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PREFACE 


As  a  result  of  the  1981  Defense  Science  Board  Summer  Study 
on  Operational  Readiness,  Task  Order  T-2-126  was  generated  to 
look  at  potential  steps  toward  improving  the  Material  Readiness 
Posture  of  DoD  (Short  Title:  R&M  Study).  This  task  order  was 
structured  to  address  the  improvement  of  R&M  and  readiness 
through  innovative  program  structuring  and  applications  of  new 
and  advancing  technology.  Volume  I  summarizes  the  total  study 
activity.  Volume  II  integrates  analysis  relative  to  Volume  III, 
program  structuring  aspects,  and  Volume  IV,  new  and  advancing 
technology  aspects. 

The  objective  of  this  study  as  defined  by  the  task  order 


"Identify  and  provide  support  for  high  payoff  actions 
which  the  DoD  can  take  to  improve  the  military  system 
design,  development  and  support  process  so  as  to  pro¬ 
vide  quantum  improvement  in  R&M  and  readiness  through 
innovative  uses  of  advancing  technology  and  program 
structure . " 

The  scope  of  this  study  as  defined  by  the  task  order  is: 

To  (1)  identify  high-payoff  areas  where  the  DoD  could 
improve  current  system  design,  development  program 
structure  and  system  support  policies,  with  the  objec¬ 
tive  of  enhancing  peacetime  availability  of  major 
weapons  systems  and  the  potential  to  make  a  rapid 
transition  to  high  wartime  activity  rates,  to  sustain 
such  rates  and  to  do  so  with  the  most  economical  use 
of  scarce  resources  possible,  (2)  assess  the  impact  of 
advancing  technology  on  the  recommended  approaches 
and  guidelines,  and  (3)  evaluate  the  potential  and 
recommend  strategies  that  might  result  in  quantum  in¬ 
creases  in  R&M  or  readiness  through  innovative  uses 
of  advancing  technology. 


The  approach  taken  for  the  study  was  focused  on  producing 
meaningful  implementable  recommendations  substantiated  by  quan¬ 
titative  data  with  implementation  plans  and  vehicles  to  be  pro¬ 
vided  where  practical.  To  accomplish  this,  emphasis  was  placed 
upon  the  elucidation  and  integration  of  the  expert  knowledge 
and  experience  of  engineers,  developers,  managers,  testers  and 
users  involved  with  the  complete  acquisition  cycle  of  weapons 
systems  programs  as  well  as  upon  supporting  analysis.  A  search 
was  conducted  through  major  industrial  companies,  a  director 


was  selected 

and 

the  following  general  plan  was  adopted. 

General  Study  Plan 

Vol. 

III 

• 

Select,  analyze  and  review  existing 
successful  program 

Vol. 

IV 

• 

Analyze  and  review  related  new  and 
advanced  technology 

Vol. 

II 

•  • 

Analyze  and  integrate  review  results 

Develop,  coordinate  and  refine  new  concepts 

Vol. 

I 

• 

Present  new  concepts  to  DoD  with  implementa¬ 
tion  plan  and  recommendations  for  application 

The 

executive 

approach 

council 

to  implementing  the  plan  was  based  on  an 
core  group  for  organization,  analysis,  inte- 

gration  and  continuity;  making  extensive  use  of  working  groups, 
heavy  military  and  industry  involvement  and  participation,  .-'nd 
coordination  and  refinement  through  joint  industry/service 
analysis  and  review.  Overall  study  organization  is  shown  in 
Fig.  P-1. 

The  basic  technology  study  approach  was  to  build  a  founda¬ 
tion  for  analysis  and  to  analyze  areas  of  technology  to  surface: 
technology  available  today  which  might  be  applied  more  broadly; 
technology  which  requires  demonstration  to  finalize  and  reduce 
risk;  and  technology  which  requires  action  today  to  provide  reli 
able  and  maintainable  systems  in  the  future.  Program  structur¬ 
ing  Implications  were  also  considered.  Tools  used  to  accomplish 
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FIGURE  P-1.  Study  Organization 

this  were  existing  documents*  reports  and  study  efforts  such  as 
the  Militarily  Critical  Technologies  List.  To  accomplish  the 
technology  studies*  sixteen  working  groups  were  formed  and  the 
organization  shown  in  Fig.  P-2  was  established. 

This  document  records  the  activities  and  findings  of  the 
Technology  Working  Group  for  the  specific  technology  as  indi¬ 
cated  in  Fig.  P-2.  The  views  expressed  within  this  document 
are  those  of  the  working  group  only.  Publication  of  this  docu¬ 
ment  does  not  indicate  endorsement  by  IDA*  its  staff*  or  its 
sponsoring  agencies. 

Without  the  detailed  efforts*  energies*  patience  and 
candidness  of  those  intimately  involved  in  the  technologies 
studied*  this  technology  study  effort  would  not  have  been 
possible  within  the  time  and  resources  available. 
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EXECUTIVE  SUMMARY 


The  maintenance  of  modern  military  systems  employs  a  variety  of 

AUTOMATION.  BUILT-lN-TEST  (BIT)  PROVIDES  ON-LINE  FAULT  DETECTION  AND 
SOME  ISOLATION,  AUTOMATIC  TEST  EQUIPMENT  (ATE)  IS  INDISPENSABLE-  AT 
INTERMEDIATE  AND  DEPOT  REPAIR  STATIONS.  AND  AUTOMATED  MAINTENANCE  AIDS 
AND  TRAINERS  ABOUND. 

These  developments  were  oesigned  to  speed  maintenance  and  to 

COMPENSATE  FOR  DECLINING  SKILL  LEVELS  IN  THE  MAINTENANCE  FORCE.  THEY 
ARE  CURRENTLY  FAR  FROM  SATISFACTORY.  MODERN  MAINTENANCE  IS 
CHARACTERIZED  BY  EXCESSIVE  FALSE  ALARMS  AND  UNNECESSARY  REMOVALS  AT  ALL 
LEVELS  OF  MAINTENANCE. 

The  RESULTS  OF  THESE  DEFICIENCIES  ARE  LONG  MAINTENANCE  TIMES. 
RESOURCES  WASTED  IN  UNNECESSARY  OR  INEFFICIENT  MAINTENANCE  ACTIONS.  AND 
SYSTEMS  OUT  OF  ACTION  WHICH  NEED  NOT  BE.  CORRECTING  THESE  PROBLEMS 
WOULD  THEREFOR  PROVIDE  BOTH  AN  ECONOMIC  ADVANTAGE  AND  A  FORCE 
MULTIPLIER. 

TO  CREATE  QUANTUM  IMPROVEMENTS  IN  MAINTENANCE  WILL  REQUIRE  THE 
APPLICATION  OF  RADICAL  CHANGES  TO  THE  TECHNOLOGY.  ONE  POSSIBILITY  IS 
THE  APPLICATION  OF  ARTIFICIAL  INTELLIGENCE  (AI)  TECHNIQUES  TO 
MAINTENANCE.  AI  IS  BEGINNING  TO  SEE  APPLICATION  TO  PRACTICAL  PROBLEMS 
IN  MANY  DISCIPLINES.  AND  HENCE  IS  POTENTIALLY  CAPABLE  OF  RELATIVELY 
RAPID  IMPLEMENTATION  INTO  MILITARY  SYSTEMS. 

AT  PRESENT,  DoO  EFFORTS  IN  APPLYING  AI  TO  MAINTENANCE  ARE  SMALL  AND 
EXPLORATORY. 

The  task  of  the  Artificial  Intelligence  Applications  committee  was 

to  EXAMINE  THE  OPPORTUNITIES  FOR  APPLYING  AI  TO  MAINTENANCE.  ASSESS  THE 
uUSTS.  RISKS.  AND  DEVELOPMENT  TIMES  REQUIRED.  AND  PROVIDE 
RECOMMENDATIONS  TO  THE  OoD  FOR  ACTION. 

The  COMMITTEE’S  RECOMMENDATIONS  ARE  DETAILED  IN  SECTION  9  OF  THIS 
REPORT.  AND  IN  THE  ATTACHED  POSITION  PAPERS.  A  SUMMARY  FOLLOWS: 

i.  The  DoD  should  take  advantage  of  the  relative  maturity  of  the 

TECHNOLOGY  FOR  CREATING  EXPERT  SYSTEMS.  SPECIFIC  APPLICATIONS  OF 
MAINTENANCE  EXPERT  SYSTEMS  SHOULD  BE  STARTED  IMMEDIATELY.  AND 
MULTI-APPLICATION  MAINTENANCE  EXPERTS  DEVELOPED  AND  STANDARDIZED. 

Develop  maintenance  expert  systems  immediately  for  current 

MAINTENANCE  APPLICATIONS  WHERE  THE  EXISTING  ATE  HAS  BEEN  INADEQUATE. 

Permit  these  systems  to  be  built  in  any  convenient  language  and 

ARCHITECTURE,  EXCEPT  THAT  TEST  PROGRAMS  GENERATED  FOR  OUTSIDE  USE  WOULD 
BE  IN  ATLAS. 


Develop  versatile  maintenance  experts  for  specific  domains  (eg 

DIGITAL  ELECTRONICS)  CAPABLE  OF  USE  IN  DIFFERENT  SYSTEMS.  SYSTEM 
SPECIFIC  DATA  WOULD  BE  REQUIRED  FOR  EACH  APPLICATION.  BUT  THE  KNOWLEDGE 
BASE  WOULD  REMAIN  THE  SAME. 

Develop  a  tool  to  automate  the  creation  of  the  system  specific  data 

REQUIRED  BY  THE  MAINTENANCE  EXPERTS  DESCRIBED  IN  THE  PRECEDING 

PARAGRAPH. 

2.  Develop  "smart"  built-in-test  (BIT)  systems  to  reduce  false  alarms, 

IDENTIFY  INTERMITTENT  FAILURES.  IMPROVE  BIT  COVERAGE. 

3.  Fund  applied  research  in  AI  for  maintenance  to  improve  expert  system 

DESIGNS  AND  TO  DEVELOP  OTHER  PROMISING  APPLICATIONS.  TOPICS  COULD 
INCLUDE  AUTOMATING  CREATION  OF  MAINTENANCE  MANUALS.  APPLICATIONS  TO 
MAINTENANCE  INFORMATION  SYSTEMS.  AI  BASED  AUTOMATIC  TEST  PATTERN 
GENERATION  ( ATPG ) .  VHSIC  DESIGN  FOR  TESTABILITY.  KNOWLEDGE  BASED 
COMPUTER  AIDED  INSTRUCTION  (CAI),  AND  SELF-IMPROVING  DIAGNOSTICS. 

M.  Foster  an  integrated  DoD-Industry  approach.  Coordinate  DoD  activity 
THROUGH  A  TRI-SERVICE  WORKING  GROUP  UNDER  THE  EXISTING  JLC  PANEL  ON 
AUTOMATIC  TESTING.  ENCOURAGE  PRIVATE  AVENUES  OF  DEVELOPMENT;  CONTINUE 
TO  SUPPORT  INDUSTRY  IR/D  IN  THE  AREA. 
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1.0.  INTRODUCTION 


The  maintenance  of  modern  military  systems  employs  a  variety  of 

AUTOMATION.  BUILT-lN-TEST  (BIT)  PROVIDES  ON-LINE  FAULT  DETECTION  AND 
SOME  ISOLATION,  AUTOMATIC  TEST  EQUIPMENT  (ATE)  IS  INDISPENSABLE  AT 
INTERMEDIATE  AND  DEPOT  REPAIR  STATIONS.  AND  AUTOMATED  MAINTENANCE  AIDS 
AND  TRAINERS  ABOUND. 

These  developments  were  designed  to  speed  maintenance  and  to 

COMPENSATE  FOR  DECLINING  SKILL  LEVELS  IN  THE  MAINTENANCE  FORCE.  THEY 
ARE  CURRENTLY  FAR  FROM  SATISFACTORY.  MODERN  MAINTENANCE  IS 
CHARACTERIZED  BY  EXCESSIVE  FALSE  ALARMS  AND  UNNECESSARY  REMOVALS  AT  ALL 
LEVELS  OF  MAINTENANCE.  EVEN  WITH  SUCCESSFUL  USE  OF  ATE.  THROUGHPUT  IS 
FAR  FROM  IDEAL.  ATE  TOO  OFTEN  FAILS  TO  ISOLATE  A  FAILURE,  REQUIRING 
SKILLFUL  HUMAN  INTERVENTION,  OR  EXPENSIVE  "SHOTGUN"  MAINTENANCE 
APPROACHES. 

The  RESULTS  OF  THESE  DEFICIENCIES  ARE  LONG  MAINTENANCE  TIMES, 
RESOURCES  WASTED  IN  UNNECESSARY  OR  INEFFICIENT  MAINTENANCE  ACTIONS.  AND 
SYSTEMS  OUT  OF  ACTION  WHICH  NEED  NOT  BE.  CORRECTING  THESE  PROBLEMS 
WOULD  THEREFOR  PROVIDE  BOTH  AN  ECONOMIC  ADVANTAGE  AND  A  FORCE 
MULTIPLIER. 

IT  CAN  BE  EXPECTED  THAT  NORMAL  EVOLUTION  OF  TESTING  TECHNOLOGY  WILL 
REDUCE  THE  SEVERITY  OF  THESE  PROBLEMS.  HOWEVER.  QUANTUM  IMPROVEMENTS 
WILL  REQUIRE  RADICAL  CHANGES  IN  APPROACHES.  ONE  POSSIBILITY  IS  THE 
APPLICATION  OF  ARTIFICIAL  INTELLIGENCE  (AI)  TECHNIQUES  TO  MAINTENANCE. 
AI  IS  BEGINNING  TO  SEE  APPLICATION  TO  PRACTICAL  PROBLEMS  IN  MANY 
DISCIPLINES.  AND  HENCE  IS  POTENTIALLY  CAPABLE  OF  RELATIVELY  RAPID 
IMPLEMENTATION  INTO  MILITARY  SYSTEMS. 

The  task  of  the  Artificial  Intelligence  Applications  committee  was 

TO  EXAMINE  THE  OPPORTUNITIES  FOR  APPLYING  AI  TO  MAINTENANCE.  ASSESS  THE 
COSTS.  RISKS,  AND  DEVELOPMENT  TIMES  REQUIRED.  AND  PROVIDE 
RECOMMENDATIONS  TO  THE  DoD  FOR  ACTION.  THIS  REPORT  PRESENTS  THEIR 
FINDIN6S  AND  RECOMMENDATIONS. 


2.0.  COMMITTEE  APPROACH  AND  MEMBERSHIP 


From  a  variety  of  sources,  28  people  were  identified  as  potential 

CONTRIBUTORS  TO  THE  STUDY.  EACH  OF  THESE  WERE  INFORMED  OF  THE  STUDY 
OBJECTIVES  AND  INVITED  TO  CONTRIBUTE  POSITION  PAPERS  DESCRIBING  THEIR 
RECOMMENDATIONS  TO  OoD  WITH  THEIR  BEST  ESTIMATES  OF  COSTS.  BENEFITS.  AND 
TECHNOLOGICAL  RISKS.  ALL  RESPONSES  WERE  CONSOLIDATED  BY  THE  COMMITTEE 
CHAIRMAN  INTO  THIS  REPORT.  CONSIDERED  AS  A  POSITION  PAPER  WAS  A  COPY  OF 
PROPOSALS  SUBMITTED  BY  THE  AIR  FORCE  HUMAN  RESOURCES  LABORATORY  TO  A 
TRI-SERVICE  WORKING  GROUP.  OTHER  INPUTS  WERE  A  SURVEY  BY  THE  ROME  AIR 

Development  Center  on  Artificial  Intelligence  applications  to 
Testability  and  various  articles  in  the  literature.  The  chairmans 

CONSOLIDATION  WAS  AIDED  GREATLY  BY  THE  TECHNICAL  ADVICE  OF  Mr.  ROBERT 
SCHRAG.  WHO  REVIEWED  EACH  ITERATION  OF  THE  DRAFT. 

The  contributors  TO  THIS  REPORT  ARE: 

o  Anthony  Coppola,  Rome  Air  Development  Center  (chairman) 

o  Eric  J. Braude.  RCA 

o  R.P.Caren.  Lockheed 

o  Marvin  Denicoff,  Office  of  Naval  Research 
o  Leonard  Friedman.  Jet  Propulsion  Laboratory 
o  Russell  M. Genet,  Air  Force  Human  Resources  Laboratory 
o  Clt  Lorraine  M.Gozzo,  Rome  Air  Development  Center 
o  John  h.  Hinchman,  General  Dynamics 
o  Robert  Hong.  Grumman 
o  Robert  Schrag,  Rome  Air  Development  Center 


3.0.  ORGANIZATION  OF  THIS  REPORT 


The  NEXT  SECTION  OF  THIS  REPORT  WILL  ILLUSTRATE  SOME  OF  THE 
PROBLEMS  WITH  CURRENT  MAINTENANCE  OF  MILITARY  EQUIPMENT.  FOLLOWING  IT 
WILL  BE  AN  INTRODUCTION  TO  ARTIFICIAL  INTELLIGENCE.  THE  NEXT  SECTION 
WILL  DISCUSS  THE  EXTENT  OF  CURRENT  APPLICATIONS  OF  AI  TO  MAINTENANCE. 
AND  THE  ULTIMATE,  THOUGH  PERHAPS  NOT  REALIZABLE.  IMPLICATIONS  OF  AI- 
TECHNOLOGY  TO  MAINTENANCE.  FOLLOWING  THIS  WILL  BE  A  DISCUSSION  OF 
CAVEATS  AND  POTENTIAL  TRAPS  IN  APPLYING  AI.  AFTER  A  BRIEF  DISCUSSION  OF 
PRESENT  AI  RESEARCH  IN  MAINTENANCE  APPLICATIONS.  THE  FINAL  SECTION  WILL 
PRESENT  A  CONSOLIDATION  OF  THE  COMMITTEES  RECOMMENDATIONS  TO  DoD. 

Individual  position  papers  by  the  committee  members  are  attached  at  the 

END  OF  THE  REPORT. 
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ILLUSTRATION  OF  MAINTENANCE  PROBLEMS 
E-3A  RADAR  TESTS 


H.O.  ILLUSTRATIONS  OF  MAINTENANCE  PROBLEMS 


FI6URE  1  SHOWS  A  SUMMARY  OF  INFORMATION  COMPILED  BY  THE  AIR  FORCE 

Test  ano  Evaluation  Center  during  tests  of  the  E-3A  (AWACS)  radar.  Of 

THE  NEARLY  12.000  INDICATIONS  OF  MALFUNCTION.  SEVEN  PERCENT  WERE 
EXCLUDED  AS  NOT  RELEVANT  TO  THE  TEST  PROGRAM  (FAILURES  CAUSED  BY 
EXTERNAL  CAUSES.  ETC.)  OF  THE  REMAINDER.  FALSE  ALARMS  OUTNUMBERED 
FAILURES  (ITEMS  ON  WHICH  MAINTENANCE  WAS  PERFORMED)  BY  MORE  THAN  TEN  TO 

one.  Although  these  were  recognized  as  false  alarms  before  starting 

THR0U6H  THE  MAINTENANCE  CHAIN,  THEY  REPRESENT  AT  THE  LEAST  A  SOURCE  OF 
ANNOYANCE  TO  A  BUSY  AIRCREW.  OF  THOSE  EIGHT  PERCENT  CONSIDERED  AS  VALID 
INDICATIONS  OF  FAILURE.  251  COULD  NOT  BE  DUPLICATED  AT  THE  FIRST 
MAINTENANCE  CHECK  (CND) .  THESE  MIGHT  BE  ADDITIONAL  FALSE  ALARMS  OR, 
PERHAPS.  INTERMITTENT  FAILURES  WHICH  OCCUR  ONLY  UNDER  THE  FLIGHT 
ENVIRONMENT.  IN  EITHER  EVENT,  25Z  OF  THE  FIRST  MAINTENANCE  ACTIONS 
ACCOMPLISHED  NOTHING. 

OF  THE  REMAINING  MAINTENANCE  ACTIONS.  981  OF  THE  FAILURES  WERE 
DETECTED  BY  THE  BIT.  LEAVING  TWO  PERCENT  TO  BE  FOUND  BY  MANUAL  MEANS. 

This  is  a  creditable  performance,  but  unfortunately  not  true  of  all 
SYSTEMS. 

When  the  attempt  was  made  to  isolate  the  failures  detected  by  BIT. 

IT  WAS  FOUND  THAT  THE  AUTOMATIC  TEST  EQUIPMENT  COULD  NOT  IDENTIFY  THE 
FAILED  COMPONENT  IN  HALF  OF  THE  CASES.  WHICH  BY  NECESSITY  REVERTED  TO 
MANUAL  PROCEDURES.  ALSO  NOTEWORTHY  IS  THAT  15Z  OF  THE  CASES  RETESTED  AS 
APPARENTLY  GOOD  (RETOK),  REQUIRING  NO  MAINTENANCE. 

The  message  of  these  figures  is  that  there  is  too  much  unnecessary 

MAINTENANCE.  THAT  THE  AUTOMATED  TECHNIQUES  ARE  CONTRIBUTING  TO 
UNNECESSARY  MAINTENANCE.  THAT  THEY  ARE  NOT  SUFFICIENTLY  REDUCING  MANUAL 
TROUBLESHOOTING.  AND.  PERHAPS,  THAT  SOME  FAILURES  ARE  ESCAPING  THE 
MAINTENANCE  PROCEDURES. 

The  false  alarm  problems  of  AWACS  are  by  no  means  unique.  An 

ON-BOARD  FAILURE  RECORDER  FOR  THE  F-111D  PROVED  INEFFECTUAL  BECAUSE  A 
SIMILAR  RATIO  OF  FAILURE  INDICATIONS  TO  ACTUAL  FAILURES.  FALSE  ALARMS 
IN  THE  F-16  CAUSED  INNOVATIONS  IN  BIT  MECHANIZATION  WHICH  WILL  BE 
DISCUSSED  LATER.  OUTSIDE  THE  MILITARY.  A  STUDY  BY  LOCKHEED  FOR  THE 

Electric  Power  Research  Institute  (1977)  found  the  same  problem  in 
Nuclear  power  plant  control  rooms.  One  rather  frightening  picture  in 

THEIR  REPORT  SHOWS  A  CONTROL  ROOM  IN  WHICH  65  WARNING  LIGHTS  COULD  BE 
SEEN.  All  WERE  IGNORED  BY  THE  OPERATORS  AS  FALSE  ALARMS. 

Another  universal  maintenance  problem  is  the  number  of  items  which 

TEST  GOOD  IN  MAINTENANCE.  FIGURES  GATHERED  FROM  THE  UNITED  STATES  AIR 

Force,  the  Canadian  Air  Force,  the  Navy  avionics  repair  facilities  and 
THE  COMMERCIAL  AIRLINES  ALL  SHOW  THAT  30Z  OR  MORE  OF  UNITS  IN 
MAINTENANCE  TEST  GOOD.  HENCE,  A  SIGNIFICANT  AMOUNT  OF  MAINTENANCE  IS 
EITHER  UNNECESSARY  OR  INEFFECTUAL. 


In  March  1983,  the  Warner-Robins  Air  Logistics  Center  provided  the 

FOLLOWING  LIST  OF  MAINTENANCE  PROBLEMS: 

o  Test  programs  proceed  in  sequence  untill  a  failure  is  found. 
Some  of  these  run  as  long  as  three  and  one-half  hours.  In  one 
OF  these,  the  most  common  failure  was  detected  in  the  last 
segments  of  the  program. 

o  Memory  devices  are  particularly  difficult  to  test. 

o  The  ATE  does  not  isolate  all  the  failures,  leaving  those  not 
found  to  the  ingenuity  of  the  maintenance  personnel. 

o  The  fault  isolation  is  too  often  ambiguous.  For  example, 
while  the  F-15  ATE  normally  isolates  a  failure  to  a  single 

PART.  IT  CAN  ALSO  RETURN  A  LIST  OF  SUSPECT  PARTS  AS  HIGH  AS 

15.  Memory  devices  are  especially  prone  to  high  ambiguity  in 

ISOLATION. 

o  The  test  program  may  isolate  to  a  string  of  components,  which 
would  all  be  replaced  or  further  isolation  performed  by  manual 
means.  Some  hybrid  circuits  cost  from  $300  to  $3,000  making 
"shotgun"  replacement  an  expensive  proposition.  Even  a  string 

OF  RELATIVELY  CHEAP  PARTS  REPRESENTS  A  SIGNIFICANT  MAINTENANCE 
COST  IN  MANHOURS  IF  ALL  ARE  REPLACED. 

o  The  ATE  will  not  isolate  chassis  problems  (eg  broken  wires) 


These  problems  translate  into  overloaded  maintenance  facilities, 

INCREASED  REQUIREMENTS  FOR  SPARE  UNITS  IN  THE  PIPELINE,  AND  EXCESSIVE 

COSTS  FOR  LABOR  AND  CONSUMABLES. 

In  SUMMARY,  THERE  IS  AMPLE  OPPORTUNITY  FOR  INCREASING  READINESS 
SIGNIFICANTLY  BY  ALLEVIATING  THESE  MAINTENANCE  DEFICIENCIES. 

Significant  improvements  in  new  systems  may  be  possible  by  merely 

DESIGNING  THE  HARDWARE  TO  BE  EASIER  TO  TEST.  ANOTHER  APPROACH, 

APPLICABLE  TO  BOTH  NEW  AND  EXISTING  SYSTEMS  IS  TO  MAKE  THE  TEST 

EQUIPMENT  "SMARTER".  SIGNIFICANT  IMPROVEMENTS  WILL  REQUIRE  SIGNIFICANT 
CHANGES  IN  TECHNOLOGY.  INCORPORATING  THE  TECHNIQUES  OF  ARTIFICIAL 

Intelligence. 


5.0.  INTRODUCTION  TO  ARTIFICIAL  INTELLIGENCE 


There  is  no  standard  definition  of  Artificial  Intelligence.  An 

OFTEN  USED  DEFINITION  IS  THE  CAPABILITY  OF  A  MACHINE  TO  DO  A  TASK  WHICH 
IF  DONE  BY  A  HUMAN  WOULD  BE  CONSIDERED  TO  REQUIRE  INTELLIGENCE.  THUS  AN 
AUTOMOBILE  WOULD  NOT  BE  AN  EXAMPLE  OF  AI .  SINCE  WALKING  IS  NOT 
CONSIDERED  TO  REQUIRE  INTELLIGENCE.  A  CHESS  PLAYING  COMPUTER.  ON  THE 
OTHER  HAND.  IS  AN  EXAMPLE  OF  AI. 

This  definition  is,  however,  too  simplistic.  Under  it.  all  BIT  and 

ATE  WOULD  BE  CONSIDERED  MANIFESTATIONS  OF  AI.  SINCE  LOCATING  FAULTS 
REQUIRES  INTELLIGENCE.  HOWEVER,  THE  BRUTE  FORCE  APPROACH  OF  A  FIXED 
SEQUENCE  OF  STIMULUS-RESPONSE  COMPARISONS  TO  PREESTABLISHED  CRITERIA 
CANNOT  BE  CONSIDERED  AN  IMPRESSIVE  SHOW  OF  INTELLIGENCE  IN  MACHINE  OR 

man.  Hence,  a  more  sophisticated  definition  is  called  for. 

To  ELIMINATE  THE  TRIVIAL  MECHANIZATIONS  FALLING  UNDER  THE  SIMPLE 
DEFINITION.  WE  SHALL  ADD  THE  CRITERIA  THAT  THE  MACHINE  MUST  HAVE  THE 
CAPABILITY  OF  FORMING  AND  MANIPULATING  ABSTRACTIONS.  THIS  WOULD  INCLUDE 
SUCH  ACTIVITIES  AS  FORMING  HYPOTHESES  (EG  THE  LOCATION  OF  A  FAILURE), 
THE  TESTING  OF  HYPOTHESES,  LEARNING.  AND  INFERENCE. 

IT  SHOULD  BE  NOTED  THAT.  FOR  OUR  PURPOSES.  IT  DOES  NOT  MATTER 
WHETHER  OR  NOT  THE  MACHINE  SOLVES  A  PROBLEM  IN  THE  SAME  MANNER  A  HUMAN 

would.  While  much  research  in  AI  is  directed  at  understanding  human 

INTELLIGENCE.  WE  WILL  BE  COMPLETELY  INDIFFERENT  TO  THE  PROCESSES 
INVOLVED.  SO  L0N6  AS  THEY  REPRESENT  THE  MOST  COST-EFFECTIVE  SOLUTIONS  TO 
THE  PRACTICAL  APPLICATIONS  OF  INTEREST. 

5.1.  AI  FIELDS  OF  STUDY 

There  is  also  no  standard  breakdown  of  AI  fields  of  study.  For 

CONVENIENCE  WE  SHALL  USE  A  LIST  OF  TOPICS  EXTRACTED  FROM  "PRINCIPLES  OF 

Artificial  Intelligence"  by  Nils  J.  Nilsson  (1980, Tioga  Publishing  Co., 
Palo  Alto.  Ca.).  In  this  section  we  will  briefly  describe  the 

OBJECTIVES  OF  EACH  TOPIC.  In  THE  NEXT  SECTION.  THEIR  CURRENT  AND 

potential  applications  to  Maintenance  will  be  discussed.  The  fields  of 

STUDY  ARE: 

5.1.1.  NATURAL  LANGUAGE  PROCESSING. 

For  our  purposes,  this  field  of  study  includes  all  attempts  to  make 
A  MACHINE  capable  of  understanding  inputs  in  natural  language  (IE 
English)  whether  typed  or  spoken.  We  shall  also  include  the  synthesis 
of  machine  replies  in  written  or  spoken  English. 

.  It  has  been  extremely  difficult  to  give  a  machine  the  capability  of 
"understanding"  even  subsets  of  .natural  language.  This  is  because  a 

MESSAGE  IS  UNDERSTOOD  NOT  ONLY  BY  ITS  TEXT.  BUT  BY  THE  TOTAL  EXPERIENCE 
OF  ITS  RECEIVER.  FOR  EXAMPLE,  CONSIDER  THE  FOLLOWING  CONVERSATION: 


"Hungry?" 


"I've  got  a  MacDonalds  coupon." 

"keys?" 

"Here.  Let's  go." 

The  many  non-spoken  portions  of  the  conversation  are  readily 
understood  by  a  human,  and  the  message  makes  sense.  To  a  machine,  it  is 

A  SEQUENCE  OF  NON-SEQUITORS . 

This  is  not  to  say  that  useful  language  recognition  has  not  been 

ACCOMPLISHED.  A  SIMULATED  ROBOT  MANIPULATOR  CALLED  SHRDLU  CAN  CARRY  ON 
MEANINGFUL  DIALOGS  IN  THE  LIMITED  SCOPE  OF  ITS  WORLD  OF  BLOCKS  ON  A 
TABLE  TOP.  KNOBS,  AN  EXPERT  SYSTEM  IN  DEVELOPMENT  FOR  TACTICAL  AIR 
MISSION  PLANNING,  HAS  AN  EXTENSIVE  NATURAL  LANGUAGE  UNDERSTANDING 
CAPABILITY. 

Recognizing  spoken  speech  compounds  the  problems  of  understanding 

THE  MEANING  OF  THE  MESSAGE  WITH  PROBLEMS  OF  RECOGNIZING  THE  MESSAGE 

itself.  Separating  words  in  spoken  sentences  has  proven  a  difficult 
chore.  Practical  use  is  being  made  of  spoken  cues  to  machines  where  a 

CUE  IS  A  ONE  WORD  DIRECTION  SUCH  AS  "BACK ", "NEXT" , OR  "TWO".  EVEN  HERE. 
THE  MACHINE  MUST  BE  TRAINED  TO  THE  VOICE  OF  ITS  OPERATOR  AND  MAY  NOT 
RESPOND  TO  ANOTHER  SPEAKER. 

Synthesising  speech  is  much  easier  than  understanding  it  as  the 

PLETHORA  OF  TALKING  COMPUTERS  INDICATES.  THE  MAIN  PROBLEMS  IN  SPEECH 
SYNTHESIS  IS  GIVING  THE  COMPUTER  THE  ABILITY  TO  KNOW  WHAT  TO  SAY. 

5.1.2.  INFERENTIAL  RETRIEVAL  FROM  DATA  BASES. 

The  design  of  efficient  data  bases  is  a  computer  science  ^ield  of 

INTEREST.  It  BECOMES  AN  AI  CONSIDERATION  WHEN  THE  DESIRED  RETRIEVAL  IS 
A  DEDUCTION  RATHER  THAN  A  STORED  FACT.  FOR  EXAMPLE,  AN  AI  MACHINE  WITH 
INTELLIGENT  RETRIEVAL  COULD  SOLVE  THE  LOGIC  PUZZLES  WHICH  PROVIDE  SUCH 

facts  as  "John  and  the  engineer  went  to  Bermuda"  and  "Marie  travelled  by 

BUS."  WITH  THE  READER  REQUIRED  TO  DEDUCE  THE  OCCUPATION  AND  VACATION 

choice  of  all  the  names  provided.  Like  language  understanding,  this 

REQUIRES  A  STORE  OF  "COMMON  KNOWLEDGE"  PERTINENT  TO  THE  PROBLEMS  OF 
INTEREST  ( E6  BERMUDA  IS  NOT  ACCESSIBLE  BY  BUS)  AS  WELL  AS  INFERENTIAL 
MECHANISMS.  A  PRACTICAL  SYSTEM  WILL  ALSO  HAVE  TO  UNDERSTAND  QUERIES. 
MAKING  NATURAL  LANGUAGE  UNDERSTANDING  ABILITY  A  VALUABLE  ADJUNCT 
FEATURE. 

5.1.3.  THEOREM  PROVING. 

There  are  several  programs  which  will  automatically  prove 

MATHEMATICAL  THEOREMS.  THE  TECHNIQUES  DEVELOPED  FOR  THESE  ARE  OF 
SIGNIFICANT  VALUE  TO  OTHER  AI  APPLICATIONS.  THEOREM  PROVING  METHODS  CAN 
BE  EXTENDED  TO  INFORMATION  RETRIEVAL  AND  FAILURE  LOCATION  WHEN  THESE 
TASKS  ARE  FORMALIZED  AS  THEOREMS  TO  BE  PROVEN.  (EG  A  THEOREM  TO  BE 
PROVEN  COULD  BE  THAT  A  FAILURE  IS  LOCATED  IN  A  GIVEN  COMPONENT.) 

Theorem-proving  techniques  attempt  to  establish  a  procedure  for 

SELECTING  FROM  AMONG  POSSIBLE  RULES  TO  APPLY  TO  THE  PROBLEM  AND  TO 


ESTABLISH  SUBPROBLEMS  LEADING  TO  THE  SOLUTION. 

5.1.4.  AUTOMATIC  PROGRAMMING. 

Strictly  speaking,  existing  compilers  are  automatic  programmers. 
They  accept  the  higher  order  language  and  write  an  object  code  to  do  a 
specified  job.  In  the  AI  sphere,  the  interest  is  in  machines  which  will 

CONVERT  HIGH  LEVEL  DESCRIPTIONS.  THE  ULTIMATE  BEING  AN  ENGLISH  INPUT.  TO 
AN  EXECUTABLE  PROGRAM.  THIS  COULD  INCLUDE  DIALOGUE  BETWEEN  MACHINE  AND 
USER  TO  RESOLVE  AMBIGUITIES.  AUTOMATIC  PROGRAMMING  SYSTEMS  CAN  ALSO 
PROVIDE  THE  VALUABLE  ADDED  BENEFIT  OF  VERIFYING  THAT  THE  PROGRAM 
PRODUCED  DOES  THE  INTENDED  JOB.  CONTRIBUTIONS  OF  WORK  IN  THE  FIELD 
INCLUDE  CONCEPTS  OF  "DEBUGGING"  AS  A  STRATEGY.  IT  CAN  BE  EASIER  TO 
MOOIFY  A  QUICKLY  GENERATED  ERRONEOUS  PROGRAM  THAN  TO  PRODUCE  A  PERFECT 
PRODUCT  ON  THE  FIRST  PASS.  ACHIEVING  THE  GOALS  OF  AUTOMATIC 
PROGRAMMING.  WILL.  HOWEVER.  REQUIRE  A  LONG  TERM  EFFORT. 

5.1.5.  COMBINATORIAL /SCHEDULING  PROBLEMS. 

The  classic  example  of  this  class  of  problem  is  the  "travelling 

SALESMAN  PROBLEM".  IN  WHICH  THE  SOLVER  ATTEMPTS  TO  FIND  A  ROUTING  WHICH 
WILL  PERMIT  A  SALESMAN  TO  VISIT  A  GIVEN  NUMBER  OF  CITIES  WITH  A  MINIMUM 
DISTANCE  TRAVELLED.  SOLUTIONS  TO  THESE  PROBLEMS  GENERATE  A 
"COMBINATORIAL  EXPLOSION"  OF  POSSIBILITIES.  THE  MOST  EFFICIENT 
SOLUTIONS  KNOWN  FOR  THIS  CLASS  OF  PROBLEM  REQUIRE  SOLUTION  TIMES  WHICH 
GROW  EXPONENTIALLY  WITH  THE  SIZE  OF  THE  PROBLEM.  AI  EFFORTS  HAVE  BEEN 
DIRECTED  TOWARDS  DELAYING  AND  MODERATING  THE  COMBINATORIAL  EXPLOSION, 
USING  KNOWLEDGE  ABOUT  THE  PROBLEM  DOMAIN.  v 

5.1.6.  MACHINE  PERCEPTION 

Understanding  of  spoken  speech  is  an  example  of  machine  perception. 
We  have  chosen  to  include  this  under  natural  language  understanding. 

SIMILARLY,  SIMULATION  of  ANY  OF  THE  HUMAN  SENSES,  AND  USE  Gi  SENSORY 
INPUTS  SUCH  AS  INFRA-RED,  RADAR  RETURNS.  ETC.,  CAN  BE  CONSIDERED  MACHINE 
PERCEPTION.  The  PR08LEMS  CAN  BE  REPRESENTED  BY  A  DISCUSSION  OF  THE 
INTERPRETATION  OF  VISUAL  IMAGES. 

Detectors  of  light  intensity  can  be  constructed.  AI  programs  exist 

TO  DEDUCE  GEOMETRIC  FEATURES  SUCH  AS  STRAIGHT  LINES  AND  TO  SEPARATE  THE 
BOUNDARIES  OF  VARIOUS  SOLID  OBJECTS.  THE  ULTIMATE  GOAL  IS  TO  PRODUCE  A 
HIGH  LEVEL  DESCRIPTION,  SUCH  AS  "A  HOUSE  WITH  THREE  WINDOWS". 

Achievement  of  this  goal  is  yet  to  come,  but  here  again  knowledge  of  the 

PROBLEM  DOMAIN  MAY  BE  THE  KEY.  (IT  SHOULD  BE  EASIER  TO  DISTINGUISH  A 
TANK  FROM  A  TRUCK  IN  A  CONVOY,  FOR  EXAMPLE.  THAN  TO  IDENTIFY  EVERY 
OBJECT  IN  A  PHOTOGRAPH  OF  TIMES  SQUARE.) 

5.1.7.  EXPERT  CONSULTING  SYSTEMS 

There  now  exist  a  number  of  automated  consultants  to  aid  in  solving 

VARIOUS  PROBLEMS,  INCLUDING  FAULT  DIAGNOSIS.  ONE  AI  TECHNIQUE  EMPLOYED 
IN  SUCH  SYSTEMS  IS  RULE  BASED  DEDUCTION.  SPECIFIC  DOMAIN  KNOWLEDGE  AND 
PROBLEM  SOLVING  RULES  OBTAINED  FROM' A  HUMAN  EXPERT  ARE  USED  BY  THE 
MACHINE  TO  FORMULATE  AND  TEST  HYPOTHESES.  THE  SYSTEM  WILL  CREATE  A 
DIALOG  WITH  ITS  USER  TO  OBTAIN  DATA  NEEDED  BY  ITS  RULES,  AND  WILL  USE 
INFERENTIAL  PROCEDURES  TO  WORK  WITH  INCOMPLETE  OR  CONFLICTING  DATA. 


Difficulties  in  creating  expert  systems  arise  in  the  reduction  of 

THE  EXPERTS'  KNOWLEDGE  TO  A  SET  OF  RULES.  EVEN  A  WILLING  EXPERT  MAY  BE 
UNABLE  TO  SO  REDUCE  HIS  KNOWLEDGE.  REPRESENTING  THE  KNOWLEDGE  BASE  IS 
ALSO  A  KEY  PROBLEM. 

Obviously,  many  developments  in  the  AI  fields  described  above  will 

HAVE  THEIR  APPLICATIONS  TO  EXPERT  SYSTEMS. 

5.1.8.  ROBOTICS 

Robots  are  the  AI  application  most  visible  to  the  public,  and 
thousands  are  in  practical  use.  However,  the  typical  application  is  far 
less  sophisticated  than  the  highly  publicized  expiremental  models.  A 
presentation  on  General  Motors  robotic  applications  at  the  1983  Annual 
Reliability  and  Maintainability  Symposium  pointed  out  that  the  robot  in 

A  WORK  STATION  IS  OFTEN  VIRTUALLY  HIDDEN  BEHIND  A  COMPLEX  OF  AUTOMATIC 
POSITIONING  MACHINERY  NEEDED  TO  ASSURE  THE  PARTS  MANIPULATED  BY  THE 
ROBOT  ARE  ALWAYS  IN  THE  EXPECTED  PLACE.  DEVELOPMENTS  IN  ROBOTICS  ARE 
AIMED  AT  CREATING  PLANNING  CAPABILITIES  TO  ELEVATE  THE  ROBOT  FROM  A  MERE 
MANIPULATOR  OF  ITEMS  IN  A  STRUCTURED  SITUATION.  AND  AT  COUPLING  MACHINE 
PERCEPTION  TO  REDUCE  ITS  DEPENDENCE  ON  AN  ORDERLY  ENVIRONMENT. 
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6.0.  CURRENT  AND  POTENTIAL  APPLICATIONS  OF  AI  TO  MAINTENANCE 


This  section  will  discuss  the  maintenance  applications  of  each  of 

THE  AI  TOPICS  DESCRIBED  ABOVE.  EXISTING  APPLICATIONS  OF  AI  IN 
MAINTENANCE  OR  IN  ANALOGOUS  FIELDS  WILL  BE  DESCRIBED.  TECHNIQUES  ON  THE 
FRINGES  OF  AI  OR  USEFUL  TO  AI  APPLICATIONS  WILL  BE  ALSO  BE  COVERED. 

Potential  near  term  applications  will  be  identified.  Finally,  the 

ULTIMATE  IMPLICATIONS  OF  THE  FIELD  OF  STUDY  WILL  BE ‘ POSTULATED .  THESE 
ARE  THE  APPLICATIONS  WHICH  COULD  BE  MADE  IF  ALL  THE  GOALS  OF  THE  AI 
FIELD  WERE  ACHIEVED.  THEY  ARE.  OF  COURSE.  FAR  OUT  BOTH  IN  TIME  AND  IN 
IMAGINATION.  AND  MAY  NEVER  BECOME  POSSIBLE.  NEVERTHELESS.  THEY 
REPRESENT  A  SET  OF  GOALS  A6AINST  WHICH  PROGRESS  CAN  BE  MEASURED'. 

6.1.  NATURAL  LANGUAGE  PROCESSING. 

AS  MENTIONED  ABOVE,  SHRDLU  CAN  CARRY  ON  A  CONVERSATION  WITH  ITS 
USER  ABOUT  ITS  LIMITED  DOMAIN.  A  MEDICAL  DIAGNOSIS  SYSTEM.  MYCIN.  CAN 
DO  THE  SAME  WITH  A  DOCTOR  ABOUT  BACTERIAL  INFECTIONS.  HENCE,  IT  IS 
CERTAINLY  A  NEAR  TERM  POSSIBILITY  THAT  A  MAINTENANCE  SYSTEM  CAN  BE  MADE 
TO  CONVERSE  WITH  ITS  USER  IN  A  USEFUL  SUBSET  OF  ENGLISH.  THIS  WOULD  BE 
ACCOMPLISHED  BY  THE  USER  TYPING  IN  HIS  SIDE  OF  THE  CONVERSATION.  THE 
MACHINE  COULD  RESPOND  BY  PRINTOUT.  CRT  DISPLAY  OR  SYNTHESISED  SPEECH. 

Spoken  interaction  would  be  highly  desirable  as  it  would  free  the 

MAINTENANCE  MAN  FROM  THE  TERMINAL.  HE  COULD  THEN  HAVE  BOTH  HANDS  FREE 
AND  COULD  OPERATE  IN  REMOTE  LOCATIONS  USING  A  PORTABLE  HEADSET  TO 
CONSULT  HIS  COMPUTER.  AT  THIS  TIME  THERE  EXIST  MAINTENANCE  TRAINERS 
WHICH  RESPOND  TO  VOICE  CUES,  WHICH  IS  CERTAINLY  A  STEP  FORWARD. 

However,  verbal  communication  in  near  conversational  style  must  be 

CONSIDERED  A  FAR  OUT  APPLICATION. 


6.2.  INFERENTIAL  RETRIEVAL  FROM  DATA  BASES 

CURRENT  APPLICATIONS  TO  MAINTENANCE  OF  THIS 
CONSIDERED  ONLY  ON  THE  FRINGES  OF  AI ,  WITH  THE  POSSIBLE 

Automatic  Test  Pattern  Generation  (ATPG).  ATPG  is  used 

TEST  PATTERNS  FOR  DIGITAL  LOGIC.  AT  PRESENT,  THE  MACHINE 
THE  HUMAN  IN  FORMULATING  TESTS  FOR  COMBINATORIAL  LOGIC, 
SEQUENTIAL  LOGIC.  NEAR  TERM  POSSIBILITIES  ARE  THE 
MACHINE  HANDLIN6  OF  SEQUENTIAL  LOGIC  AND  DEVELOPMENT  OF 
CIRCUITRY.  THE  ULTIMATE  POTENTIAL  WOULD  BE  THE  COMPLETE 
THE  MANUAL  PART  OF  THE  PROCESS. 


TOPIC  MUST  BE 
EXCEPTION  OF 
TO  GENERATE  THE 
IS  SUPERIOR  TO 
AND  INFERIOR  IN 
IMPROVEMENT  OF 
ATPG  FOR  ANALOG 
ELIMINATION  OF 


On  THE  FRINGE  OF  AI  IS  THE  USE  OF  COMPUTERIZED  MODELS  TO  DIRECT  THE 
TESTING  PROCESS.  THERE  ARE  THREE  EXISTING  PROGRAMS  WHICH  WILL  USE  A 
STORED  REPRESENTATION  OF  AN  EQUIPMENT  TO  REDUCE  THE  NUMBER  OF  TESTS 
NEEDED  TO  ISOLATE  A  FAILURE.  THE  FIRST  OF  THESE  WAS  LOGMOD  (DETEX 
Corp.).  The  LOGMOO  DATA  base  is  a  USER  PREPARED  LOGICAL  model  OF  THE 
SYSTEM.  USIN6  THIS  MODEL,  A  HEURISTIC  (RULE  FOLLOWING)  PROCEDURE  USES 
THE  RESULTS  OF  EACH  TEST  MADE  BY  THE  MAINTENANCE  MAN  TO  DIRECT  HIM  TO 
THE  NEXT  TEST  SUCH  THAT  EACH  TEST  ELIMINATES  HALF  OF  THE  SUSPECT 


to 


to 
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COMPONENTS  UNTIL  THE  FAILED  UNIT  IS  ISOLATED.  THE  STAMP  PROGRAM  (ARINC) 
USES  A  NODAL  REPRESENTATION  OF  THE  SYSTEM  AND  PROVIDES  VARIOUS  SEARCH 
OPTIONS  SUCH  AS  ELIMINATING  COMPONENTS  WITH  HALF  THE  PREDICTED  FAILURE 
RATE.  THE  THIRD  SYSTEM,  FIND,  IN  DEVELOPMENT  BY  HUGHES,  HAS  SIMILAR 
FEATURES. 

The  Modular  Automatic  Test  Equipment  (MATE)  office  of  the  Air  Force 
Aeronautical  Systems  Division  has  created  a  working  example  of  a  self¬ 
improving  DIAGNOSIS  (SID)  SYSTEM.  THIS  SYSTEM  WOULD  BE  USED  WHEN  A 
FAILURE  CANNOT  BE  FOUND  BY  ATE.  IT  WOULD  SUMMON  MANUAL  TROUBLESHOOTING 
AND  REQUEST  A  RECORD  OF  THE  SUCCESSFUL  REPAIR  ACTION.  THESE  RECORDS 
WOULD  BE  USED  TO  RECOMMEND  REPAIR  ACTIONS  ON  FUTURE  OCCURRENCES  OF  THE 
SAME  FAILURE  SYMPTOM.  WHILE  THE  TEST  MODEL  USES  ONLY  THE  RELATIVE 
FREQUENCY  OF  SUCCESSFUL  FIXES  FOR  A  PARTICULAR  SYMPTOM,  INFERENTIAL 
MECHANISMS  COULD  BE  ADOED  TO  DETECT  TRENDS  WHICH  SUGGEST  REVISIONS  TO 
THE  STRATEGY  DICTATED  BY  FREQUENCY  ALONE. 

Also  proposed  by  the  MATE  office  is  the  recording  by  the  SID  of  the 

SERIAL  NUMBERS  OF  ALL  ITEMS  REPAIRED  AND  THE  APPLICATION  OF  HEURISTICS 
TO  IDENTIFY  ITEMS  WHICH  RETURN  TOO  OFTEN.  THESE  WOULD  BE  FLAGGED  AS 
REQUIRING  SPECIAL  ATTENTION  BECAUSE  OF  THE  POSSIBILITY  OF  A  CHRONIC 
PROBLEM  NOT  SOLVED  BY  THE  ROUTINE  PROCEDURE.  THIS  FEATURE  NOW  EXISTS  IN 

a  Marconi  ATE  system. 

6.3.  THEOREM  PROVING 

Theorem  proving  techniques,  as  mentioned,  are  used  in  AI 

APPLICATIONS  IN  THE  OTHER  FIELDS  OF  INTEREST  LISTED,  PARTICULARLY  IN 
EXPERT  SYSTEMS.  THEIR  CONTRIBUTIONS  IN  THESE  APPLICATIONS  WILL  BE 
COVERED  IN  THE  APPROPRIATE  DISCUSSIONS.  THERE  IS  ONE  APPLICATION  WHICH 
IS  WELL  WORTH  NOTING  HERE. 

BIT  SIGNALS  ARE  TOO  OFTEN  FALSE  ALARMS.  IN  ADDITION,  INTERMITTENT 
FAILURES  ARE  DIFFICULT  TO  DISTINGUISH  FROM  FALSE  ALARMS.  THEOREM 
PR0VIN6  TECHNIQUES  COULD  BE  EMPLOYED  TO  TEST  BIT  SIGNALS  TO  DISCRIMINATE 
AGAINST  FALSE  ALARMS  AND  TO  IDENTIFY  INTERMITTENT  FAILURES. 

A  SIMPLE  PROCEDURE  WAS  USED  BY  THE  F- 16  TO  EVALUATE  BIT  SIGNALS. 

Each  signal  was  stored  and  the  BIT  checked  again  after  a  short  time.  If 

FIVE  OUT  OF  SEVEN  CHECKS  AGREED  ON  A  FAILURE  INDICATION,  THE  BIT  LATCH 
WAS  TRIGGERED.  SMART  BIT  WOULD  EXTEND  THIS  TO  SUCH  ACTIONS  AS  CHANGING 
THE  DECISION  THRESHOLD,  ADJUSTING  THE  BIT  SENSITIVITY,  AND  IDENTIFYING 
INTERMITTENT  FAILURES.  SMART  BIT  SYSTEMS  ON  INDIVIDUAL  WEAPONS  SYSTEMS 
COULD  COMPARE  THEIR  MEMORIES  TO  IDENTIFY  UNUSUAL  CIRCUMSTANCES.  THE 
COMPARISON  MIGHT,  FOR  EXAMPLE,  SHOW  ONE  SYSTEM  WITH  MORE  THAN  EXPECTED 
FAILURE  INDICATIONS  INDICATING  SOME  PROBLEM  WITH  THE  PLATFORM  (EG  A 
FAULTY  ENVIRONMENTAL  CONTROL  SYSTEM).  WITH  INCORPORATED  ENVIRONMENTAL 
DATA,  SMART  BIT  COULD  HELP  LOCATE  AN  IDENTIFIED  INTERMITTENT  FAILURE. 

The  Boeing  OBIT  circuit  and  the  Battelle  stress  meter  are  designed  to 

RECORD  ENVIRONMENTAL  DATA  AND  COULD  BE  ADDED  TO  THE  SMART  BIT.  SMART 
BIT  IS  CONSIDERED  A  NEAR  TERM  POSSIBILITY. 

6.M.  AUTOMATIC  PROGRAMMING 

ATPG,  MENTIONED  ABOVE, CAN  BE  CONSIDERED  AN  EXAMPLE  OF  AUTOMATIC 
PROGRAMMING.  THERE  ARE  ALSO  PROGRAMS  WHICH  ASSIST  A  TEST  ENGINEER  IN 
CREATING  ATLAS  STATEMENTS  FOR  TEST  EQUIPMENT.  THOUGH  THESE  ARE  NOT 


REALLY  SOPHISTICATED  ENOUGH  TO  WARRANT  THE  TITLE.  THE  ULTIMATE 
APPLICATION  OF  AUTOMATIC  PROGRAMMING  WILL  BE  THE  REAL  TIME  GENERATION  OF 
ATLAS  TEST  VECTORS  FROM  A  COMPUTERIZED  REPRESENTATION  OF  THE  SYSTEM.  AS 
THEY  ARE  NEEDED  BY  AN  EXPERT  SYSTEM  ISOLATING  A  FAILURE.  THIS  IS 
DEFINITELY  NOT  A  SHORT  TERM  GOAL. 

6.5.  COMBINATORIAL /SCHEDULING  PROBLEMS 

WORK  IN  THIS  TOPIC  CAN  BE  USED  IN  THE  NEAR  TERM  TO  DEVELOP  MODELS 
FOR  QUEUING  MAINTENANCE  FOR  MINIMUM  IMPACT  ON  SYSTEM  DOWN  TIME.  IT  CAN 
ALSO  BE  USED  TO  DESIGN  FOR  FAULT  TOLERANCE  OF  DISTRIBUTED  SYSTEMS  SUCH 
AS  COMMAND  AND  CONTROL  SYSTEMS  OR  FOR  FAULT  TOLERANCE  IN  VHSIC  CHIPS. 

The  ultimate  potential  of  the  techniques  would  be  in  real  time 

MAINTENANCE  SCHEDULING  AND  IN  REAL  TIME  RECONFIGURATION  OF  NETWORKS 
AROUND  A  FAILED  COMPONENT. 

6.6.  MACHINE  PERCEPTION 

Current  AI  programs  can  identify  shapes.  On  a  simpler  level,  the 
Marconi  ATE  mentioned  above  will  read  serial  numbers  marked  in  bar 
codes,  and  optical  character  readers  are  available  to  read  stylized 

ALPHANUMERICS.  WE  CAN  THUS  EASILY  PRODUCE  SYSTEMS  WHICH  WILL  READ 
INFORMATION  PRINTED  ON  A  BOARD.  BUT  THIS  WILL  BE  OF  NO  USE  UNTIL  THE  ATE 
IS  PROGRAMMED  TO  MAKE  USE  OF  THE  INFORMATION.  IN  THE  NEAR  TERM  WE  CAN 
ADD  AUTOMATION  TO  ATE  LETTING  IT  CALL  ITS  OWN  TEST  PROGRAMS  FROM  ITS 
IDENTIFICATION  OF  THE  BOARD.  THIS  COULD  BE  OF  VALUE  IN  THAT  IT  WOULD'. 
ELIMINATE  SOME  OF  THE  DULLER  RESPONSIBILITIES  OF  THE  MAINTENANCE  MAN. 
IN  THE  LONG  TERM.  MACHINE  PERCEPTION  CAN  BE  USED  TO  GET  FULL  VALUE  FROM 
ROBOT  SYSTEMS  BY  REC06NIZING  ROTATIONS  AND  DISPLACEMENTS  OF  ITEMS  TO  BE 
TESTED.  WHICH.  WITH  AN  APPROPRIATE  CONTROL  PROGRAM,  WILL  ELIMINATE  THE 
NEED  FOR  ITEMS  TO  BE  FED  TO  THE  ROBOT  IN  A  RIGIDLY  CONTROLLED  MANNER. 

Presuming  a  robot  fed  ATE  system,  the  automatic  identification  features 

WOULD  BECOME  QUITE  USEFUL,  AS  THE  HUMAN  WOULD  BE  NEEDED  ONLY  WHEN  THE 
MACHINE  REQUIRED  INTELLIGENT  HELP.  WITH  AMBULATORY  ROBOTS,  PERCEPTION 
COULD  BE  USED  TO  LOCATE  PARTICULAR  BOARDS  IN  AN  EQUIPMENT  FOR  REMOVAL  BY 
THE  ROBOT  AND  CONVEYANCE  TO  THE  ATE. 

6.7.  EXPERT  CONSULTING  SYSTEMS 

Expert  systems  are  in  practical  use  today  for  configuring  computer 

SYSTEMS  (Rl),  ANO  ARE  AVAILABLE  FOR  MEDICAL  DIAGNOSIS  (MYCIN .CADUCEUS) 
AND  LOCATING  MINERAL  DEPOSITS  (PROSPECTOR).  THERE  ARE  ALSO  EXPERT 
SYSTEMS  IN  VARIOUS  STATES  OF  DEVELOPMENT  FOR  FAULT  ISOLATION.  THESE 
INCLUDE  DELTA  FOR  THE  MAINTENANCE  OF  LOCOMOTIVES.  REACTOR  FOR  NUCLEAR 
REACTORS.  CRITTER  FOR  DIGITAL  CIRCUITS,  IDT  USED  ON  A  COMPUTER.  AND  DART 
DESI6NED  FOR  COMPUTER  HARDWARE.  (SEE  ALSO  POSITION  PAPER  BY  CAREN  ON  A 
GENERIC  MAINTENANCE  EXPERT.  LES).  HENCE,  MAINTENANCE  EXPERT  SYSTEMS  ARE 
RELATIVELY  NEAR  TERM  POSSIBILITIES. 

The  diagnostic  search  programs  ( LOGMOD, STAMP. F IND ) .  mentioned 

ABOVE,  WOULD  SEEM  TO  BE  A  VALUABLE  ADJUNCT  TO  AN  EXPERT  MAINTENANCE 
PROGRAM  IN  THAT  THEY  CAN  BE  USED  BY'  THE  PROGRAM  TO  SET  UP  ITS  STRATEGY 
FROM  THE  DESIGN  OF  THE  UNIT  UNDER  TEST. 


An  expert  systems  requires  a  knowledge  base  of  its  domain.  At 

PRESENT.  THIS  IS  PROVIDED  BY  THE  USER.  IQGMOD,  ET  AL.  REQUIRE  USER 
LAYOUT  OF  THE  SYSTEM  AND  USER  GENERATION  OF  APPROPRIATE  TESTS.  THE 
ULTIMATE  EXPERT  SYSTEM  WOULD  USE  ADVANCES  IN  AUTOMATIC  PROGRAMMING  TO 
PERFORM  FULLY  AUTOMATIC  TESTING  BY  DERIVING  ITS  KNOWLEDGE  FROM  A  MACHINE 
READ  SCHEMATIC.  FORMING  ITS  STRATEGY  FROM  THAT  KNOWLEDGE.  AND  GENERATING 
APPROPRIATE  TESTS  AS  IT  NEEDS  THEM. 

Once  the  ATE  has  the  knowledge  it  needs  to  operate  as  an  expert 

SYSTEM,  IT  CAN  ALSO  BE  USED  AS  A  MAINTENANCE  AID  TO  DIRECT  A  HUMAN  WHEN 
HIS  INTERVENTION  IS  NECESSARY.  IT  COULD  ALSO  PROVIDE  THE  INTERACTION 
NEEDED  FOR  TRAINING  ITS  OPERATOR.  THUS  THE  EXPERT  SYSTEM  WOULD  SERVE  AS 
ATE.  MAINTENANCE  AID.  AND  TRAINING  DEVICE.  WHILE  PROBABLY  NOT  A  SHORT 
TERM  EFFORT.  THE  INTEGRATION  OF  MAINTENANCE  AIDS  SHOULD  NOT  BE  A  FAR  OUT 

proposition.  Integrating  a  training  capability  will  be  a  long  term 

PROPOSITION  NEEDING  ADVANCES  IN  COMPUTER  AIDED  INSTRUCTION  (CAI) 
TECHNIQUES. 

Computer  aided  design  (CAD)  programs  can  be  considered  a  form  of 

EXPERT  SYSTEM.  EVEN  THOUGH  THEIR  CURRENT  APPROACHES  MAY  NOT  BE 
CONSIDERED  TO  BE  EXAMPLES  OF  AI .  THE  ROME  AIR  DEVELOPMENT  CENTER  IS 
STUDYING  THE  INCORPORATION  OF  EASE  OF  TEST  CONSIDERATIONS  TO  CAD 
PROGRAMS.  From  THESE  CAN  SPRING  AN  EXPERT  SYSTEM  WHICH  WHICH  WILL 
CREATE  A  DESIGN  WITH  A  MINIMUM  OF  MAINTENANCE  PROBLEMS.  THIS  WILL 

REQUIRE  THE  CREATION  OF  A  SET  OF  DESIGN  RULES  FOR  EASE  OF  MAINTENANCE 

AND  A  SET  OF  RULES  TO  TRADE  THESE  OFF  AGAINST  OTHER  CONSIDERATIONS  SUCH 
AS  THERMAL  DESIGN,  WIRING  CONSTRAINTS.  ETC.  THE  DESIGN  RULES  SHOULD  BE 
A  NEAR  TERM  POSSIBILITY,  BUT  CREATING  THE  TRADE-OFF  RULES  WILL  BE  A  MORE 
DIFFICULT  TASK. 

6.8.  ROBOTICS 

Current  use  of  robots  is  mostly  as  programmable  manipulators.  A: 

LEAST  TWO  COMPANIES  HAVE  DESIGNED  ROBOT  MANIPULATORS  TO  FEED  TEST 
SAMPLES  INTO  ATE.  ANOTHER  IMMEDIATE  POSSIBILITY  IS  THE  HANDLING  OF  THE 
ADAPTERS  WHICH  FORM  THE  INTERFACE  BETWEEN  TEST  SAMPLES  AND  THE  ATE. 

RELIEVING  THE  HUMAN  OF  THE  NEED  TO  MANUALLY  CHANGE  ADAPTERS.  NEAR  TERM 

POSSIBILITIES  ARE  THE  USE  OF  MACHINE  PERCEPTION  PERMITTING  THE  ,r;OBOT  TO 
WORK  IN  AN  UNORGANIZED  ENVIRONMENT  (EG  CORRECTLY  FEED  PRINTED  CIRCUIT 
BOARDS  PILED  RANDOMLY  ON  A  TABLE.)  THE  FAR  OUT  APPLICATION  IS  THE  USE  OF 
AN  AMBULATORY  ROBOT  WITH  AN  EXPERT  SYSTEM  TO  PERFORM  COMPLETELY 
AUTOMATIC  MAINTENANCE. 


7.0.  CAVEATS 


Herbert  Oreyfus.  a  critic  of  AI.  divides  intelligent  activity  into 

FOUR  CLASSES.  HlS  LOWEST  CLASS.  ASSOCIATIONISTIC  (LEARNED  BY  MEMORY) 
INCLUDES  SUCH  EXAMPLES  AS  MAZE  PROBLEMS.  THE  NEXT  HIGHER  CLASS.  SIMPLE 
FORMAL  (LEARNED  BY  RULE)  INCLUDES  GAMES  LIKE  TIC-TAC-TOE  AND  THE  PROOF 
OF  THEOREMS  USING  MECHANICAL  PROOF  PROCEDURES.  THESE  PROBLEMS  ARE 
EASILY  HANDLED  BY  AI  TECHNIQUES.  THE  HIGHEST  CLASS,  NONFORMAL 
ACTIVITIES,  (LEARNED  BY  EXAMPLE)  INCLUDES  PROBLEMS  WHICH  DREYFUS  SEES  NO 
POSSIBILITY  FOR  SOLUTION  BY  AI  TECHNIQUES.  THESE  INCLUDE  NATURAL 
LANGUAGE  TRANSLATION  AND  ILL-DEFINED  GAMES  SUCH  AS  RIDDLES.  THE 
REMAINING  CLASS  IS  COMPLEX  FORMAL  (LEARNED  BY  RULE  AND  PRACTICE)  WHICH 
INCLUDES  UNCOMPUTABLE  GAMES  LIKE  CHESS  AND  PROOF  OF  THEOREMS  WHfcRE  NO 
MECHANICAL  PROOF  PROCEDURE  APPLIES.  HERE  AI  IS  MOST  DIFFICULT  TO  APPLY 
AND  THIS  IS  THE  REALM  OF  MAINTENANCE  PROBLEMS.  FORTUNATELY.  AS  THE  MANY 
CHESS-PLAYING  PROGRAMS  ATTEST.  THE  DIFFICULTIES  ARE  NOT  INSURMOUNTABLE. 

AS  WITH  ALL  NEW  TECHNOLOGY.  APPLICATIONS  OF  AI  CANNOT  BE  MADE 
CARELESSLY.  BESIDES  THE  NATURAL  LIMITATIONS  AND  TECHNICAL  CONSTRAINTS, 
WE  MUST  CONSIDER  THE  IMPACT  OF  AI  ON  THE  MAINTENANCE  PERSONNEL.  WE 
SHALL  NOW  BRIEFLY  DISCUSS  SOME  POTENTIAL  TRAPS. 

The  NATURAL  LIMITATIONS  OF  AI  ARISE  FROM  THE  FACT  THAT  WE  CANNOT 
BUILD  A  MACHINE  WHICH  DUPLICATES  THE  INTELLIGENCE  OF  A  HUMAN.  HENCE.  WE 
CANNOT  CREATE  A  MACHINE  WHICH  WILL  UNDERSTAND  NATURAL  LANGUAGE.  BECAUSE 
WE  CANNOT  BUILD  IN  THE  TOTAL  EXPERIENCE  THAT  A  HUMAN  USES  TO  INTERPET  A 
MESSAGE.  WE  CAN  ONLY  HOPE  TO  GIVE  IT  SUFFICIENT  ABILITY  TO  COMMUNICATE 
IN  ITS  DOMAIN  AND  SOME  CAPABILITY  TO  ADD  NEW  WORDS  TO  ITS  VOCABULARY. 

IT  WILL  BE  A  LONG  TIME.  IF  EVER,  THAT  A  MACHINE  WILL  HAVE  A  TRUE 
ABILITY  TO  LEARN.  NO  PLANS  SHOULD  ANTICIPATE  SUCH  A  CAPABILITY  UNTIL 
EVIDENCE  OF  SIGNIFICANT  PROGRESS  IS  AVAILABLE. 

IN  CREATIN6  EXPERT  SYSTEMS  BY  REDUCING  THE  EXPERTS  OPERATION  TO  A 
SET  OF  RULES,  WE  IGNORE  THE  FACT  THAT  EXPERTISE  IS  A  BEHAVIOR  RATHER 
THAN  A  SET  OF  RULES.  IF  THE  RULES  WILL  SUFFICE,  THE  SYSTEM  WILL  BE 
SUCCESSFUL.  IF  NOT,  WE  MUST  PROVIDE  FOR  THE  HUMAN  TO  TAKE  OVER  AS 
NECESSARY. 

Technical  constraints  on  AI  include  processing  time  and  memory 

REQUIREMENTS.  AI  PROGRAMS  ARE  NOTED  FOR  FILLING  UP  LARGE  MACHINES. 

While  higher  speeds  ano  larger  memories  are  still  being  developed. 

PRACTICAL  APPLICATIONS  MUST  RECOGNIZE  THE  LIMITS.  AS  AN  EXAMPLE.  TEN 
FACTORIAL  COMBINATIONS  IS  NOT  A  LARGE  NUMBER.  YET  IF  THE  ANALYSIS  OF 
EACH  TOOK  23  MILLISECONDS,  THE  PROBLEM  WOULD  REQUIRE  2*4  HOURS  OF  MACHINE 
TIME.  AS  ANOTHER  EXAMPLE,  THERE  ARE  APPROXIMATELY  10  TO  THE  50TH  POWER 
ATOMS  IN  THE  EARTH.  THIS  IS  FAR  SMALLER  THAN  THE  POSSIBLE  NUMBER  OF 
MOVES  IN  A  CHESS  GAME.  HENCE,  THE  COMBINATORIAL  EXPLOSION  IS  SOMETHING 
TO  BE  AVOIDED  IN  AI  PROGRAMS.  AS  THE  CHESS-PLAYERS  DO  BY  PRUNING  THEIR 
SEARCH  TREES. 


A I  PROGRAMS  ARE  TYPICALLY  BEGUN  AS  SMALL  FEASIBILITY 
DEMONSTRATIONS.  TROUBLE  BEGINS.  HOWEVER.  WHEN  ONE  ATTEMPTS  TO  SCALE  UP 
THE  PROGRAM  TO  HANDLE  REAL  WORLD  COMPLICATIONS.  HENCE.  THE  TECHNICAL 
LIMITS  TO  AN  AVAILABLE  TECHNIQUE  MUST  BE  CONSIDERED  BEFORE  ITS 
APPLICATION  TO  A  NEW  PROBLEM. 

Finally,  in  applying  AI  to  maintenance  we  must  consider  the 

MAN-MACHINE  RELATIONSHIP.  DECLINING  SKILL  LEVELS  PROMOTE  A  TEMPTATION 
TO  FOLLOW  A  "SMART  MACHINE-DUMB  MAN"  PHILOSOPHY.  IF  THE  MAN  IS  SUBJECT 
TO  DIRECTION  BY  A  MACHINE  WHICH  OOES  NOT  CREDIT  HIM  WITH  ANY  CAPABILITY. 
THEN: 


o  Any  capability  he  has  is  wasted 
0  He  will  not  improve  in  capability 
0  HE  UILL  FIND  NO  satisfaction  with  his  job 


The  items  listed  above  are  not  only  demeaning  to  the  man,  they  can 

CREATE  A  DANGEROUS  WORKING  CLIMATE. 

For  this  reason,  AI  applications  to  maintenance  should  to  the 

GREATEST  EXTENT  POSSIBLE  BE  DESIGNED  TO  ADAPT  TO  THE  SKILL  OF  THE  USER. 
AND  SERVE  AS  A  MEANS  TO  IMPROVE  HIS  SKILL.  THERE  ARE  CONSIDERATIONS 
WHICH  CAN  BE  MADE.  INTELLIGENT  TRAINERS  NOW  EXIST  WHICH  WILL  ADAPT 
THEIR  MODE  OF  OPERATION  TO  THE  NEEDS  OF  THE  STUDENT.  THERE  IS  NO  NEED 
FOR  AN  EXPERT  SYSTEM  TO  COMMUNICATE  BY  A  FIXED  LIST  OF  QUESTIONS  TO  THE 
OPERATOR.  IT  CAN  BEGIN  BY  ASKING  FOR  THE  OPERATORS  FINDINGS  AND 
CONCLUSIONS. 

AI  SYSTEMS  MUST  TO  THE  EXTENT  POSSIBLE  BE  DESIGNED  SO  THAT  THE 
HUMAN  WILL  CONSIDER  IT  AS  A  PARTNER  RATHER  THAN  AS  AN  INANIMATE  TYRANT 
FOR  WHICH  THE  HUMAN  PERFORMS  TRIVIAL  FUNCTIONS. 


8.0.  PRESENT  RESEARCH  IN  AI  APPLICATIONS  TO  MAINTENANCE 


A  ROUSH  ESTIMATE  OF  FY-83  FUNDING  IN  AI  RESEARCH  IS  THE  RANGE  FROM 
$36  MILLION  TO  $46  MILLION.  OF  THIS.  DoD  AGENCIES  ACCOUNT  FOR  $20 
MILLION.  AND  OTHER  U.S.  AGENCIES.  $6  MILLION.  THE  MAJORITY  OF  THESE 
FUNDS  ARE  6.1.  INDUSTRY  WILL  SPEND  AN  ESTIMATED  $10-20  MILLION.  WHICH 
CAN  BE  CONSIDERED  MOSTLY  6.2 

Of  these  funds.  We  have  identified  $0.3  million  being  spent  by  DoD 

AGENCIES  IN  STUDIES  OF  AI  APPLICATIONS  TO  MAINTENANCE.  AND  THIS  AMOUNT 

IS  SPREAD  EQUALLY  AMONG  STUDIES.  THESE  ARE: 

■nrA.ee 

(1) .  A  STUDY  BY  THE  ROME  AIR  DEVELOPMENT  CENTER  TO  DETERMINE  THE 
OPPORTUNITIES  AND  RISKS  OF  AI  APPLICATIONS  TO  TESTABILITY.  AWARDED 

April.  1983  to  Boeing. 

(2) .  A  STUDY  BY  THE  NAVAL  AlR  ENGINEERING  CENTER  TO  DEVELOP  AI 
APPLICATIONS  TO  NAVY  ATE.  AWARD  PENDING. 

(3) .  A  STUDY  BY  THE  AIR  FORCE  HUMAN  RESOURCES  LABORATORY  TO  ADAPT 
MEDICAL  EXPERT  SYSTEMS  TO  MAINTENANCE.  AWARDED  FEB.  1983  TO  SYSTEMS 
Exploration. 

In  addition,  there  are  some  other  efforts,  such  as  an  AFHRL  study 

OF  AI  APPLICATIONS  TO  TRAINING.  WHICH  WILL  PROVIDE  RESULTS  USEFUL  TO 
MAINTENANCE. 

In  FY-84.  The  MATE  office  will  continue  its  work  on  Self-improving 
Diagnostics,  which  was  unfunded  in  FY-83.  RADC  will  begin  an  effort  to 
develop  designs  for  Smart  BIT.  The  Air  Force  Institute  of  Technology 
will  be  working  with  RADC  and  Warner-Robins  Air  Logistic  Center  to 
develop  an  experimental  expert  system  for  depot  maintenance,  aimed  at 
SPECIFIC  PROBLEMS  OF  V^ALC. 

There  is  a  significantly  greater  amount  of  Industry  IR/D  directed 
AT  APPLYING  AI  TO  MAINTENANCE.  A  ROUGH  ESTIMATE  IS  $3  MILLION  IN  FY'83. 
THIS  DOES  NOT  INCLUDE  CAPITAL  INVESTMENTS,  SUCH  AS  GE'S  DEVELOPMENT  OF 
DELTA  TO  AID  IN  LOCOMOTIVE  REPAIR. 

IN  SUMMARY,  CURRENT  DOD  EFFORTS  ARE  SMALL  AND  EXPLORATORY. 


9.0.  RECOMMENDATIONS  TO  THE  DEPARTMENT  OF  DEFENSE 


Despite  the  fact  that  the  members  of  the  committee  worked 

COMPLETELY  INDEPENDENTLY.  THERE  IS  ONLY  ONE  AREA  OF  SIGNIFICANT 
DISAGREEMENT  IN  THE  POSITION  PAPERS.  THIS  IS  THE  RECOMMENDED  LANGUAGE 
IN  WHICH  AI  PROGRAMS  SHOULD  BE  WRITTEN.  AN  AREA  OF  GENERAL  AGREEMENT 
WAS  THAT  EXPERT  CONSULTANT  SYSTEMS  CAN,  AND  SHOULD.  BE  APPLIED  TO 
MAINTENANCE  NOW.  THERE  WAS  EVEN  A  GENERAL  CONSENSUS  OF  THE  DEVELOPMENT 
RESOURCES  REQUIRED  FOR  AN  EXPERT  SYSTEM:  TWO  YEARS  TIME.  $200,000  IN 
COMPUTER  COSTS.  AND  FIVE  TO  TEN  MAN-YEARS  PER  YEAR.  HOWEVER. 
DISCUSSIONS  AFTER  REVIEW  OF  THE  POSITION  PAPERS  WOULD  CAUSE  THESE  TO  BE 
CONSIDERED  MINIMUM  PROJECTIONS.  WITH  PERHAPS  DOUBLE  THE  COMPUTER 
RESOURCES  AND  FIVE  YEARS  OF  TIME  REQUIRED.  THE  FOLLOWING 
RECOMMENDATIONS  ARE  THE  CHAIRMAN'S  CONSOLIDATION  OF  THE  POSITION  PAPERS, 
DISCUSSIONS  WITH  THE  RADC  CONTRIBUTORS.  AND  HIS  DERIVATIONS  FROM  THE 
INFORMATION  GIVEN  ABOVE.  ALL  THE  POSITION  PAPERS  ARE  ATTACHED  TO  THIS 
REPORT.  THE  READER  IS  ENCOURAGED  TO  REVIEW  THEM  FOR  FURTHER  DETAILS. 

RECOMMENDATION  NO.  1 

The  DoD  should  take  advantage  of  the  relative  maturity  of  the 

TECHNOLOGY  FOR  CREATING  EXPERT  SYSTEMS.  SPECIFIC  APPLICATIONS  OF 
MAINTENANCE  EXPERT  SYSTEMS  SHOULD  8E  STARTED  IMMEDIATELY.  AND  MULTI¬ 
APPLICATION  MAINTENANCE  EXPERTS  DEVELOPED  AND  STANDARDIZED. 

The  DoD  should  immediately  develop  expert  systems  for  existing 

MAINTENANCE  APPLICATIONS  WHERE  MAINTENANCE  IS  PARTICULARLY  TROUBLESOME. 

As  an  example,  the  AFIT-RADC-WRALC  program  would  attempt  to  create  a 

SYSTEM  TO  WORK  WITH  THE  F- 15  ANALOG  PRINTED  CIRCUIT  BOARD  TEST  STATION. 
IT  WOULD  FIRST  BE  PROGRAMMED  WITH  THE  KNOWLEDGE  REQUIRED  TO  TROUBLESHOOT 
ONLY  ONE  BOARD.  THE  MOST  TROUBLESOME  OF  THOSE  THE  ATE-  HANDLES.  THIS 
WOULD  SHOW  THE  VALUE  OF  THE  APPROACH  AND  PERMIT  DEBUGGING  OF  THE  SYSTEM. 

More  knowledge  would  be  added  incrementally  until  the  system  handled 

EVERY  BOARD  ASSIGNED  TO  THE  ORIGINAL  ATE.  AT  THIS  POINT,  IT  WOULD 
HOPEFULLY  BE  COST  EFFECTIVE  TO  SCRAP  THE  ORIGINAL  SYSTEM.  IF  NOT.  THE 
EXPERT  SYSTEM  WOULD  STILL  EARN  ITS  KEEP  BY  ITS  SUPERIOR  HANDLING  OF 
PROBLEM  BOARDS.  EACH  SERVICE  COULD  PICK  A  PROMISING  CANDIDATE  (A  SYSTEM 
WHICH  IS  NOT  HANDLED  WELL  BY  THE  ATE.  AND  FOR  WHICH  EXPERT  MAINTENANCE 
PERSONNEL  ARE  BOTH  AVAILABLE  AND  WILLING  TO  COOPERATE  IN  CREATING  THE 
EXPERT  MAINTENANCE  SYSTEM).  AS  IT  BUILDS  AND  REFINES  THE  MAINTENANCE 
EXPERT  SYSTEM.  THE  SERVICE  WOULD  IMPROVE  THE  OPERATIONAL  READINESS  OF 
THE  CANDIDATE  SYSTEM  WHILE  IT  GAINS  EXPERIENCE  AND  CONFIDENCE  WITH  THE 
AI  TECHNOLOGY.  NO  RISK  WOULD  BE  INVOLVED,  SINCE  THE  EXISTING  ATE  WOULD 
STILL  BE  IN  PLACE.  RESOURCES  WOULD  BE  TWO  TO  FIVE  YEARS  CALENDAR  TIME. 
10-20  MANYEARS  OF  EFFORT  AND  $200,000  TO  $500,000  IN  COMPUTER  COSTS  FOR 
EACH  SYSTEM.  EACH  SYSTEM  WOULD  PAY  FOR  ITSELF  IN  SHORT  ORDER,  BY 
REDUCIN6  MAINTENANCE  TIME  AS  MUCH  AS  501.  HOWEVER.  THE  REAL  VALUE  OF 
THESE  FIRST  EFFORTS  WOULD  BE  IN  THE  KNOWLEDGE  GAINED.  (SEE  POSITION 

papers  by  Friedman,  Braude,  Denicoff,  Hong.  Caren,  Genet) 


TO  PERMIT  IMMEDIATE  APPLICATION.  THE  FIRST  AI  MAINTENANCE  SYSTEMS 
SHOULD  BE  BUILT  IN  THE  LANGUAGE  AND  ARCHITECTURE  MOST  CONVENIENT  TO  THE 
BUILDER  AND  USER.  WITH  A  BLANKET  EXEMPTION  FROM  ANY  CURRENT  POLICIES  ON 
LANGUAGES.  THE  ONLY  EXCEPTION  WOULD  BE  THAT  ANY  TEST  SEQUENCE  GENERATED 
BY  THE  SYSTEM  FOR  OUTSIDE  USE  WOULD  BE  IN  ATLAS.  NO  COST  INVOLVED. 

Will  cause  a  proliferation  of  languages  for  first  systems,  but  will 

PERMIT  EARLIER  IMPLEMENTATION.  BY  YEARS.  AND  PROVIDE  INFORMATION  NEEDED 
FOR  ULTIMATE  STANDARDIZATION.  (CHAIRMAN'S  RECOMMENDATION  BASED  ON 
CONFLICTING  INPUTS  IN  POSITION  PAPERS  BY  FRIEDMAN.  HlNCHMAN,  AND  CAREN) 

TO  IMPROVE  COST-EFFECTIVENESS  IN  THE  LONGER  TERM.  THE  DoD  SHOULD 
DEVELOP  VERSATILE  MAINTENANCE  EXPERTS  FOR  SPECIFIC  DOMAINS.  SUCH  AS 
DIGITAL  ELECTRONICS.  WHICH  ARE  USED  IN  MANY  DIFFERENT  SYSTEMS.  THEY 
WOULD  CONTAIN  THE  NECESSARY  THEORY  AND  DIAGNOSTIC  STRATEGIES  FOR  THEIR 
SPECIFIC  DOMAINS.  THEY  MUST  BE  USER  FRIENDLY  (INTERACT  IN  A  SUBSET  OF 

English,  explain  their  actions,  and  adapt  to  the  skill  of  the  user). 

AND,  PRESUMING  PROGRESS  IN  COMPUTER  AIDED  INSTRUCTION  (CAI)  TECHNIQUES. 
EACH  SYSTEM  COULD  ULTIMATELY  SERVE  AS  AN  INTEGRATED  ATE.  MAINTENANCE 
TRAINER  AND  TRAINING  AID.  ONE  BASIC  SYSTEM  (FOR  ONE  DOMAIN)  COULD  BE 
BUILT  IN  TWO  YEARS  WITH  10  MANYEARS  EFFORT.  SYSTEM  SPECIFIC  DATA  BASES 
WOULD  BE  INCORPORATED  DURING  THE  DEVELOPMENT  OF  THE  SYSTEMS  TO  BE 

tested.  Refinements  would  be  added  as  developed.  Benefit  would  be  the 

ELIMINATION  OF  THE  NEED  FOR  REINVENTING  THE  ENGINE  FOR  EVERY 
APPLICATION.  EASILY  WORTH  MILLIONS  IN  DEVELOPMENT  AND  TRAINING  SAVINGS. 

Technical  risk  is  moderate.  (See  position  papers  by  Caren.  Braude. 
Friedman) 

Further  improvements  in  cost-effectiveness  would  be  made  possible 

BY  DEVELOPING  A  SYSTEM  BUILDING  TOOL  TO  AUTOMATE  THE  CREATION  OF  THE 
SYSTEM  SPECIFIC  DATA  REQUIRED  BY  THE  EXPERT  SYSTEM  DISCUSSED  IN  THE 
PRECEDING  PARAGRAPH.  THE  TOOL  WOULD  EXTRACT  THE  NEEDED  KNOWLEDGE  EITHER 
FROM  A  HUMAN  EXPERT  OR,  IDEALLY,  FROM  A  DESCRIPTION  OF  THE  SYSTEM  TO  BE 
TESTED.  This  WILL  MINIMIZE  ONE  OF  THE  MAJOR  COSTS  OF  THE  EXPERT  SYSTEM. 

Cost  would  be  about  $200,000  a  year  in  computer  costs  and  ten  manyears 

PER  YEAR.  A  PROTOTYPE  COULD  BE  AVAILABLE  IN  TWO  YEARS.  BUT  IT  MIGHT 
TAKE  A  FIVE  YEAR  PROGRAM  TO  COMPLETE  A  SUPPORTABLE  PRODUCT.  BENEFIT 
WOULD  BE  SIGNIFICANT  SAVINGS  IN  TIME  AND  ELIMINATION  OF  ERRORS  FOR  EVERY 
NEW  SYSTEM  TO  WHICH  IT  IS  APPLIED.  NO  MORE  THAN  FIVE  APPLICATIONS.  IF 
THAT  MUCH,  WOULO  REPAY  THE  COSTS  WITH  A  DIVIDEND  IN  EARLIER  TEST  SYSTEM 
AVAILABILITY  AND  EASIER  MODIFICATION  AS  THE  DESIGN  OF  THE  SYSTEM  UNDER 
TEST  CHANGES.  TECHNICAL  RISK  IS  PRESENTLY  CONSIDERED  HIGH.  (SEE 
POSITION  PAPERS  BY  HlNCHMAN.  HONG.) 

NOTE:  THE  EXPERT  SYSTEMS  WOULD  ELIMINATE  THE  LONG  TEST  PROGRAMS 
NOW  USED  IN  CONVENTIONAL  ATE.  THIS  FEATURE  CAN  BE  INCORPORATED  INTO 
CONVENTIONAL  ATE  SYSTEMS.  TO  DO  SO  THE  DoD  COULD  PROHIBIT  ALL  NEW  ATE 
SYSTEMS  TO  USE  INFLEXIBLE  SEQUENTIAL  TEST  PROCEDURES.  INSTEAD  REQUIRE 
THE  USE  OF  SEGMENTED  TEST  PROGRAMS  WHICH  ARE  CALLED  OUT  IN  THE  ORDER 
NEEDED  FOR  MOST  RAPID  FAULT  ISOLATION  USING  THE  STRATEGIES  NOW  AVAILABLE 
IN  LOGMOD.  STAMP.  AND  FIND.  COST  WILL  BE  A  SIGNIFICANT  INCREASE  IN 
EFFORT  REQUIRED  TO  PROGRAM  THE  ATE  AND  SOME  ADDITIONAL  MEMORY.  WILL 
PROBABLY  PERMIT  THE  ELIMINATION  OF  ONE  MAINTENANCE  SHIFT,  PAYING  FOR 
ITSELF  IN  ONE  YEAR  OR  TWO.  (CHAIRMAN'S  RECOMMENDATION) 


RECOMMENDATION  NO.  2 


Develop  "smart"  BIT  for  digital  electronic  systems  to  minimize 

FALSE  ALARMS.  IDENTIFY  INTERMITTENT  FAILURES,  IMPROVE  COVERAGE  OF  BIT. 

An  RADC  proposed  FY-84  effort  hopes  to  provide  design  concepts  which 

COULD  BE  USED  BY  INDIVIDUAL  DESIGNERS  TO  CONSTRUCT  SMART  BIT  IN  THEIR 
PARTICULAR  APPLICATIONS.  A  COMPLETE  SERIES  OF  STUDIES  LEADING  TO  THE 
DESIGN  OF  AN  ON-BOARD  KNOWLEDGE  BASED  MONITORING  SYSTEM  OR  THE  DESIGN 
AND  TEST  OF  EXPERIMENTAL  BIT  SYSTEMS  COULD  RUN  TWO  TO  FOUR  YEARS  AND  ONE 
TO  SIX  MILLION  DOLLARS.  BENEFITS  ARE  INCALCULABLE  SINCE  THEY  INCLUDE 
THE  WORTH  OF  REDUCED  MISSION  ABORTS  DUE  TO  FALSE  ALARMS.  MORE  TANGIBLE 
BENEFITS  COULD  BE  A  901  REDUCTION  IN  FALSE  ALARMS.  AND  THE  DECREASE  OF 
THE  PORTION  OF  UNITS  SENT  TO  REPAIR  WHICH  TEST  GOOD.  FROM  THE  PRESENT 
30X  TO  PERHAPS  10X.  A  SUCCESSFUL  APPLICATION  SHOULD  PAY  FOR  ITSELF  IN 
TWO  YEARS  OF  OPERATION  ON  ONE  SYSTEM.  AND  PROVIDE  A  MEASURABLE 
IMPROVEMENT  IN  THE  READY  RATE  OF  THE  SYSTEM  USING  IT.  (SEE  POSITION 

papers  by  Hong.  Braude.) 

RECOMMENDATION  NO.  3 

Fund  applied  Research  and  Development  of  AI  for  maintenance,  both 
to  improve  the  capabilities  of  maintenance  expert  systems  and  to  apply 

AI  TO  OTHER  MAINTENANCE  APPLICATIONS.  Some  SPECIFIC  TOPICS  ARE: 

1.  Automating  the  creation  and  presentation  of  Technical  Manuals. 
(See  position  paper  by  Denicoff.) 

2.  Applying  AI  to  Maintenance  Information  Systems  and  databases. 
(See  position  papers  by  Braude,  Denicoff.  Genet.  Hong.) 

3.  Developing  crisis  alerting  systems.  (See  position  paper  by 
Denicoff.) 

M.  For  expert  maintenance  systems,  developing  requirements  for 

LANGUAGES  AND  COMPUTER  SYSTEMS,  TECHNIQUES  FOR  IMPROVING  USER 
FRIENDLINESS.  AND  MORE  SOPHISTICATED  APPROACHES  (EG  MEANS  OF  FORMING 
RULES  FROM  THE  CIRCUIT  ITSELF  RATHER  THAN  FROM  AN  EXPERT  FAMILIAR  WITH 

the  circuit).  (See  position  papers  by  Braude.  Hong.) 

5.  Developing  AI  systems  for  Automatic  Test  Program  Generation 
(ATPG).  The  current  AI  programs  used  to  develop  test  patterns  for 

DIGITAL  COMBINATORIAL  LOGIC  SHOULD  BE  EXTENDED  TO  SEQUENTIAL  LOGIC  AND 
ANALOG  CIRCUITS.  SYSTEMS  SHOULD  WORK  FROM  THE  CIRCUIT  DESCRIPTION  AND 
PROVIDE  TEST  VECTORS  IN  ATLAS.  (SEE  POSITION  PAPERS  BY  HONG,  BRAUDE.) 

6.  Applying  AI  techniques  to  VLSI.VHSIC  design  for  fault  tolerance 

AND  TESTABILITY.  THIS  SHOULD  BE  INCORPORATED  INTO  THE  VHSIC  PHASE  THREE 
STUDY  PLANS.  (SEE  POSITION  PAPER  BY  HONG. ) 

7.  Developing  knowledge  based  computer  aided  instruction  (CAI) 

SYSTEMS  FOR  MAINTENANCE  TRAINING.  (SEE  POSITION  PAPER  BY  HONG. )  NOTE: 
THIS  COULD  ULTIMATELY  8E  INCORPORATED  INTO  THE  ATE  ITSELF. 


8.  Developing  self-improving  diagnostics  and  test  program  sets. 
(See  position  paper  by  Braude,  and  discussion  of  MATE  effort  in 
PARAGRAPH  6.2  ABOVE.) 
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Other  topics  will  be  identified  by  the  FY-83  studies  begun  by  RADC. 
NAEC,  AFHRL.  Recommend  6.2  PROGRAMS  be  started  by  all  three  services. 
FUNDED  AT  ONE  MILLION  DOLLARS  PER  SERVICE  PER  YEAR.  TO  BEGIN  WORK. 

Promising  developments  should  be  followed  by  6.3  projects  with 

APPROPRIATE  HIGHER  FUNDING. 

RECOMMENDATION  NO.  4 

Foster  an  integrated  DoD-Industry  approach. 

Coordinate  the  various  efforts  of  DOD  agencies  through  a 
tri-service  group  on  AI  applications  to  maintenance.  Recommended  group 
would  8E  a  committee  under  the  JtC  Automatic  Testing  Panel.  It  could 

ALSO  BE  UNDER  JDL  WORKING  GROUP  FOR  AI.  BUT  SEEMS  MORE  APPROPRIATE  FOR 

the  Automatic  Testing  Panel  because  of  its  interface  with  other 

COMMITTEES.  PARTICIPANTS  WOULD  INCLUDE  ALL  SERVICE  AGENCIES  INVOLVED  TO 
SHARE  RESPONSIBILITIES  AND  AVOID  DUPLICATION  OF  EFFORTS.  IT  WOULD  ALSO 
PROVIDE  A  CONTACT  POINT  FOR  DoD  AND  INDUSTRY.  (CHAIRMAN'S 
RECOMMENDATION) 

Encourage  private  avenues  of  development  of  AI  applications  to 

MAINTENANCE:  CONTINUE  TO  SUPPORT  INDUSTRIAL  IR/D  IN  THE  AREA.  EXPRESS 
DOD  INTEREST  AT  APPROPRIATE  MEETINGS,  PROVIDE  COPIES  OF  THIS  REPORT  TO 

Industry.  The  NSIA  Testability  Committee,  which  parallels  the  JLC 

PANEL.  SHOULD  BE  ENCOURAGED  TO  CREATE  A  SUBGROUP  ON  AI  APPLICATIONS  TO 
SERVE  AS  AN  INDUSTRY  FOCAL  POINT.  THE  CLOSE  WORKING  RELATIONSHIP  OF  THE 
NSIA  COMMITTEE  AND  THE  JLC  PANEL  WOULD  BE  A  NATURAL  AVENUE  FOR  CREATING 
A  DIALOG  ON  AI  APPLICATIONS.  (CHAIRMAN'S  RECOMMENDATION.) 
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RELIABILITY  AND  MAINTAINABILITY 
POSITION  PAPER 

INTRODUCTION 

Many  important  aspects  of  Reliability  and  Maintainability  require  reasoning 
about  incompletely  defined  circumstance,  often  with  inconsistent  evidence. 
Artificial  Intelligence  provides  for  the  computeri  ation  of  tasks  of  this 
type;  as  well  as  providing  an  "understanding"  interface  to  the  more 
mechanical  tasks,  allowing  the  power  of  the  machines  to  be  more  effectively 
employed  in  the  quest  for  improved  R&M.  Three  areas  of  potential  applications 
are  outlined. 

1.  DELIVERING  TROUBLESHOOTING  EXPERT  SYSTEMS 

CONCURRENTLY  WITH  THE  SYSTEMS _ 

In  many  cases  when  a  contractor  develops  equipment,  the  contractor  accumulates 

a  data  base  of  experience  with  failure  modes,  critical  maintenance  paths,  and 

reliability  data  which  is  supposed  to  be  captured  in  BITE  (built-in  test 

equipment)  and  in  maintenance  manuals.  Much  of  this  expertise  and  the 

associated  rules -of -thumb  could  be  captured  in  an  expert  system.  In 

addition,  there  are  unacceptable  false  alarm  rates  (e.g.  40%)  associated  with 

built-in  test  equipment  (BITE).  The  particular  applicability  of  expert 

system  technology  to  the  false  alarm  problem  Is  explored  further  in  position 

paper  #5. 

a.  Recommended  DoD  Action 

Study  the  feasibility  of  requiring  from  contractors  the  following  type 
of  Item:  an  adaptable,  easily  expandable  expert  system  (computer  program) 
whose  purpose  It  Is  to  trouble-shoot  the  equipment  being  delivered  by 
the  contractor. 

Eric  J.  Braude 
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1.  The  DoD  should  experiment  wi th  retro-fitting  this  idea  to  some 
sample  existing  systems,  including  attention  to  false  alarms, 
ii.  Assuming  favorable  results,  the  DoD  should  require  from  sample 
projects  a  paper  version  of  an  expert  system  applicable  to  the 
iima intena nee  of  the  delivered  item.  Some  questions:  at  what 
maintenance  level  are  expert  system  approaches  useful;  how  are 
they  to  be  specified? 

A  cost  comparison  should  be  made  to  allow  cost/benefit  analyses  of 
approaches  which  integrate  more  or  less  expert  system  technology, 
ill.  The  above  step  could  be  repeated  for  a  new  project,  but  the 
knowledge  formatting  requirements  could  be  more  formal  so  as 
to  make  them  closer  to  those  that  would  be  required  by  a  real  expert 
system  knowledge  base.  An  important  question  to  be  addressed  is, 

"how  should  the  expertise  be  collected?", 
iv.  In  parallel  with  the  above  three  efforts,  appropriate  expert  system¬ 
building  tools  could  be  identified.  Special  attention  could  be 
given  to  make  the  knowledge  acquisition  process  easily  accessible 
by  a  contractor  not  necessarily  familiar  with  expert  systems, 
v.  In  its  final  state  of  maturity,  the  contractor  could  be  paid  to 
deliver  a  working  expert  system  at  the  same  time  as  the  deliverable 
equipment.  The  expert  system  could  then  be  run  concurrently  with 
operational  use  of  the  equipment  which  it  maintains. 
c.  Estimated  Time  to  Completion  and  Cost  (Very  Rough) 

Stages  (1)  through  (Iv)  -  two  years  for  projects  at  $200K  each.  The  last 
phase  would  cost  approximately  5-15%  of  the  cost  of  the  project  itself. 
Much  would  depend  on  the  efficacy  of  the  expert  system  building  tools. 
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(1)  Advantages 

Significant  decrease  in  the  span  between  system  delivery  and 
operational  readiness  maturity. 

Improved  understanding  of  the  system  under  use. 

(2)  Disadvantages 

Increased  initial  cost  in  system  development 
e.  Technological  Risk 

o  Possibility  that  expert  system  technology  may  fail  as  a  solution 
to  the  maintainability  problem. 

o  Possibility  that  current  and  projected  technology  cannot  achieve 
expert  system  building  tools  which  are  usable  by  system  developers, 
o  An  underestimation  of  percentage  increase  in  costs  and  schedule 
associated  with  the  expert  system  add-on  effort. 

2.  AUTOMATIC  TEST  PROGRAM  GENERATION _ 

The  generation  of  test  programs— whether  to  be  run  on-line  or  off  line— is 
extremely  time-consuming,  expensive  and  error  prone.  New  ways  must  be  found 
to  generate  test  programs. 
a.  Recomwended  DoO  Action 

The  DoO  should  encourage  research  and  development  in  methods  to  generate 
test  programs  based  on: 

(1)  formal  descriptions  of  the  unit  under  test,  or 

(2)  functional,  performance,  and  anticipated  failure  modes  of  the  unit. 
These  inputs  would  be  processed  by  an  independent  processor  which 
generate  a  suite  of  test  programs  for  the  unit. 
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(1)  A  study  should  be  commissioned  to  collect  the  results  of  existing 
approaches  to  this  kind  of  problem. 

12)  The  existing  efforts  should  be  analyzed  to  identify  missing 
capabilities.  R&D  should  be  funded  to  address  these  missing 
capabilities. 

(3)  Efforts  should  be  funded  to  try  out  the  most  promising  methods  on 
existing  systems  in  laboratory  environments  so  as  to  compare  them 
with  test  program  generation  methods. 

(4)  Several  knowledge-based  processors  of  the  type  described  should  be 
commissioned  by  the  government  for  use  in  common  areas  such  as  the 
testing  of  standard  kinds  of  circuits. 

(5)  These  knowledge-base  processors  will  become  GFE  on  contracts 
requiring  test  program  generation. 

c.  Estimated  Time  to  Completion  and  Cost  (Very  Rough) 

Year  1  S300K 

2  5700K 

3  J1.5M 

4  S1.5M 

d.  Impact  On  Operational  Readiness 

(1)  Advantages 

Considerable  saving  to  the  government  in  costs  of  programming  test 
programs. 

Reduction  In  time  wasted  running  Irrel event  tests.  Ability  to  cope 
with  uncertainty  (especially  In  analog  testing). 

(2)  Disadvantages 

Risk  of  developing  a  new  programming  system  and  cost  of  running  it. 
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o  Uncontrolled  costs  in  knowledge-base  acquisitions 
o  Inability  to  adequately  describe  functionality  of  equipment  within 


feasible  knowledge-base  technology 

o  Waste  due  to  lack  of  experience  in  trading  off  costs  of  hand-coded 
tests  versus  expert  system  construction  costs. 


3.  ADAPTABLE  SPARING  SYSTEM 


Readiness  is  heavily  influenced  by  sparing  procedures.  Sparing  database 
technology  is  often  utilized.  In  reality,  however,  many  factors  affect 
sparing  which  are  currently  not  considered  computerizable;  such  as 
aircraft  availability,  bottlenecks,  and  weather  crises. 
a.  Recommended  DoD  Action 

The  DoD  should  encourage  the  use  of  data  base  mechanisms  which  allow 
for  uncertainty  factors.  There  are  several  efforts  under  way,  especially 
within  the  academic  community,  to  insert  a  level  of  intelligence  between 
the  user  and  the  data  base.  For  example,  in  an  initial  state,  a  part 
might  be  readily  available  from  an  expert  depot.  As  time  progresses, 
however,  the  system  could  permit  the  entering  of  uncertainty  factors 
which  would  degrade  the  availability  of  the  part. 


fciH7 


ested  Approach 


This  would  be  almost  identical  to  2b. 
c.  Estimated  Time  to  Completion  aiid  Cost 


This  would  be  almost  identical  to  2c. 
Impact  on  Operational  Readiness 


(1)  Advantages 

The  Insertion  of  layers  of  intelligence  will  have  the  effect  of 
automating  more  of  the  human  effort  required  in  sparing.  This 
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leaves  readiness  personnel  more  time  to  consider  factors  which 
are  known  to  be  beyond  the  scope  of  the  decision  aid  system. 

(2)  Disadvantages 

Changes  like  the  ones  proposed  always  cause  doubts  about  the  system1 
value  because  they  leave  to  machines  those  functions  which  are 
traditionally  performed  by  humans.  The  responsibility  for  decisions 
is  thereby  weakened. 
e.  Technological  Risk 

Even  though  systems  approaching  these  capabilities  have  been  built, 
their  free- form  nature  leads  to  doubts  about  whether  such  systems  can 
be  properly  bounded  or  even  specified.  In  addition,  much  work  needs  to 
be  done  on  understanding  commonality  across  domains.  For  example,  can 
a  readiness  system  for  one  aircraft  be  easily  modified  to  serve  another? 

4.  AI  APPLICATIONS  IN  BITE  MAN-MACHINE  INTERFACE _ 

The  complexity  of  Built-In  Test  Equipment  (BITE),  as  well  as  the  near-term 
application  of  (knowledge-based)  expert  system  principles  to  BITE,  will 
require  the  ability  to  effectively  exchange  information  across  a  "friendly" 
man/machine  interface.  Artificial  Intelligence  techniques  can  help  make 
this  Interface  friendlier  by  embedding  a  layer  of  intelligence  and/or 
natural  language  facilities. 

a.  Recommended  DoO  Action 

DoD  must  address  the  extension  of  human  engineering  to  system  complexity 
at  the  man/machine  Interface  believed  to  be  a  significant  factor  in 
personnel  recruitment,  training,  and  retention  problems  In  all  of  the 
services. 

b.  Suggested  Approach 
Almost  Identical  with  1b. 
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Estimated  Time  to  Completion  and  Cost 


Similar  to  1c. 

d.  Impact  on  Operational  Readiness 
(1)  Advantages 

A  well-designed  synergistic  man/machine  interface  will  be  unambiguous 
to  all  of  the  system's  users,  regardless  of  role,  background,  or 
level  of  expertise  (e.g.  low  aptitude  vs.  high  aptitude;  urban  vs.  rural 
background;  novice  vs.  expert,  etc.).  The  system  will  tailor  its 
service  requests  and  explanatory  capabilities  by  adapting  to  the 
user's  level  of  understanding.  Training  will  be  an  inherent  part  of 
the  military  system  (tank,  EW  analysis  center,  ATE,  etc.)  because  it 
will  act  as  both  a  skill  enhancer  and  retainer  when  not  being  operated 
in  its  primary  role.  All  of  these  factors  will  foster  a  greater 
acceptance  of  the  system(s)  by  the  user  community  because  they  will 

be  tolerant*of  human  error,  adapt  to  style  and  need,  be  capable  of 

* 

giving  the  user  tHfe  perception  that  he/she  is  in  control,  and  allow 

✓ 

the  user  to  easily  visualize  an  operationally  meaningful  model  of 
the  processes  present  in  the  system. 

(2)  Disadvantages 

An  Improved  man/machine  interface  will  effect  both  a  greater  initial 
system-development  cost,  as  well  as  a  greater  per-system  cost  due  to 
the  use  of  graphic  displays,  and  so  on.  (These  costs  are  dwarfed  by 
the  advantages  afforded  by  the  Improved  Interface  that  Is  being  proposed.) 

e.  Technological  Risk 

There  Is  no  perceived  technological  risk  in  applying  hunan  engineering  to 
these  systems.  (A  much  greater  loss  will  take  place  If  this  technology 
Is  not  utilized  because  eventually  one  may  be  left  with  mammoth  systems 
that  are  unusable  because  of  their  complexity.  ) 


a 
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5.  APPLICATION  OF  ARTIFICIAL  INTELLIGENCE  TO  THE  FALSE  ALARM  PROBLEM 
An  unacceptably  high  false  alarm  rate  (e.g.  40  per  cent)  exists  in  many 
systems  which  incorporate  Bui  It- In-Test-Equipment  (BITE)  according  to  the 
OSB  1981  Summer  Study  Panel  on  Operational  Readiness  with  High  Performance 
Systems.  Indeed,  any  system  of  sufficient  complexity  is  nearly  certain  to 
generate  false  alarms  requiring  human  intervention.  There. is,  therefore,  a 
need  to  use  diagnostic  aids  with  the  greatest  naturalness  so  that  fault 
indicators  and  their  soundness  can  be  understood.  (Conventional  approaches 
typically  require  the  presence  of  a  quite  knowledgeable  trouble-shooter  to 
identify  false  alarms.  Several  existing  expert  systems  have  demonstrated  an 
unmatched  ability  to  answer  questions  with  the  appropriate  level  of  detail 
(MYCIN,  AL/X,  etc.)  and,  thereby,  accommodate  varying  levels  of  user  skill. 

a.  Recommended  DoD  Action 

The  OoO  should  encourage  research  and  development  in  improving  the 
“explanation  subsystem"  of  expert  systems  with  an  emphasis  towards 
ease  of  use  Including: 

(1)  efforts  towards  Improving  current  models  of  the  level  of 
sophistication  of  the  user, 

(2)  modeling  what  the  particular  user  has  learned  already, 

(3)  closer  accounting  of  the  time  spent  searching  answers  to  various 
critical -path  questions,  and 

(4)  anticipation  of  user  questions  based  on  past  sessions  so  that 
near-critical-path  questions  will  be  anticipated  and  some 
preliminary  work  performed. 

b.  Suggested  Approach 

(1)  A  study  should  be  comnlssloned  to  collect  the  results  of  existing 
approaches  to  constructing  user  models  and  employing  these  models 
to  perform  work  on  anticipated  questions. 
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(2)  Existing  efforts  should  be  analyzed  to  identify  missinq  capabilities. 
R&D  should  be  funded  to  address  these  missing  capabilities. 

(3)  Efforts  should  be  funded  to  try  out  the  most  promising  methods  on 
existing  systems  in  laboratory  environments. 

(4)  New  explanatory  BITE  systems  should  be  funded. 

c.  Estimated  Time  to  Completion  and  Cost  (Very  Rough) 

Year  1  $300K 

2  $400 K 

3  $1M 

4  S1-3M 

d.  Impact  on  Operational  Readiness 

( 1 )  Advantages 

Discovery  of  serious  false  alarms  in  a  shorter  amount  of  time  and 
far  faster  isolation  of  the  source  of  these  alarms  to  prevent 
recurrence. 

Includes  self  maintenance  derived  from  knowledge  based  expert  system 
principles.  The  Built-In-Test  would  support  the  maintainer  in 
troubleshooting,  accept  his  conjecture  as  to  the  problem  cause, 
suggest  additional  tests,  explain  the  rationale  for  the  test 
sequence  and  final  diagnosis.  Note  that  this  is  another  way  to 
embed  training. 

(2)  Disadvantages 

The  possibility  of  excessive  use  of  computer  resources  spent  modeling 
potential  user  questions. 


Eric  J.  Braude 
RCA 
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Sometimes  too  verbose. 
e.  Technological  Risk 

o  difficulty  of  user-modeling 

o  few  or  no  existing  systems  have  had  to  model  near-critical 
path  question/answering. 


Eric  J.  Braude 
RCA 
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Mr.  Anthony  Coppola 
PA^/BBST 

Griffiss  AFB,  NY  13441 

Dear  Tony:  10  March  1983 

t  have  been  discussing  the  application  of  Artificial 
Intelligence  to  improve  the  materiel  readiness  posture 
with  several  FCA  people  over  the  last  few  weeks. 

The  position  paper  which  T  submitted,  entitled 
"Reliability  and  Maintainability"  is  my  transcription 
of  discussions  which  T  held  with  Tom  Carver  of  PCA's 
Automated  Systems  in  'urlington,  Mass.  Tom's  draft 
of  that  discussion  was  delayed  in  our  internal  mail, 
and  so  T  did  not  have  time  to  reconcile  his  transcription 
with  mine,  t  am,  therefore,  enclosing  it  with  this 
letter  in  the  expectation  that  it  will  usefully  contribute 

to  the  project. 

/ 

A  sixth  position  paper,  entitled,  "Self  Adaptive  Test 
Program  Sets"  is  also  submitted  herewith. 

Sincerely, 

Eric  J.  Braude,  Manager 
Software  Technology  Laboratory 

/ab 

Copies  to  Mr.  Anthony  T.  Feduccia 
Mr.  Tames  R.  Feller 


4  a.  DEVELOP  ATE  MAN-MACHINE  INTERFACE  WHICH  ADAPTS  TO  USER  SKILL  LEVEL 


a.  Recommended  DoD  Action 

Develop  an  efficient  operator  interface  which  recognizes  that  ATE 
operators  represent  a  wide  sprectrum  of  skill  levels  and  that  this 
Interface  is  a  source  of  errors  contributing  to  reduced  machine 
availability. 

b.  Suggested  Approach 

The  man-machine  interface  for  typical  ATE  systems  is  a  CRT  and 
keyboard.  This  interface  is  used  to  control  the  ATE ,  define  the 
test  problem,  guide  manual  intervention  in  the  test  procedure  and 
maintain  the  ATE.  The  1981  Defense  Science  Board  Study  Panel  on 
Operational  Readiness  expressed  the  following: 

"System  complexity  at  the  man-machine  interface  (particularly 
involving  the  maintainer)  is  causing  personnel  recruitment, 
training  and  retention  problems  in  all  the  services". 

Suggested  approach  is  to: 

-  Define  the  information  exchange  across  the  ATE  man-machine 
interface. 

-  Develop  the  range  of  interaction  needed  to  operate  and  maintain 
the  ATE. 

-  Develop  approaches  for  more  efficient  communication  of  the 
above  information/interaction. 

-  Examine  current  and  projected  display/input  technology  for 
application  to  this  problem. 

-  Explore  approaches  to  sense  operator  skill  level  and  modify  the 
Interface  accordingly. 

-  Modify  an  existing  ATE  interface  for  use  as  a  test  bed. 

As  a  control  experiment  modify  the  ATE  power  up/power  down 
sequences  to  incorporate  the  candidate  man-machine  approaches. 

-  Modify  the  selected  ATE  self-test  diagnostic  logic  to 
Incorporate  new  approaches  and  lessons  learned  for  first 
experiment. 

c.  Estimated  Time  to  Completion  Cost 

One  year  of  accomplished  power  up/power  down  sequence,  one  year  to 
accomplish  revised  ATE  self  test  experience.  Step  1  will  required 
$20K  material,  two  man-years  engineering.  Step  2  would  required 
$10K  material/computer  time  and  two  man-years  engineering. 
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d. 


Iwpact  on  Operational  Readiness 
(1)  Advantages 


-  Positive  impact  on  personnel  recruiting,  training  and 
retention 


-  Reduces  operator  errors 

-  Improved  ATE  availability 

-  Improved  test  system  thru-put 
(2)  Disadvantages 

-  Probable  increase  in  ATE  unit  cost. 
Technological  Risk 


Nan-machine  interface  may  be  too  subtle  to  accomplish  the 
machine  hardware/software  adaptation  to  skill  levels.  Nay 
be  necessary  to  define  some  number,  -  e.g.,  three  to  five  of 
specific  skill  levels. 


Hardware  implementation  of  improved  man-machine  interface 
may  pose  reliability  problems  in  military  field  environment, 
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SELF  ADAPTIVE  TEST  PROGRAM  SETS 


Recommended  DoD  Action 


Develop  ATE  application  software  which  can  learn  from  test 
experince  to  improve  test  and  diagnostic  efficiency. 

Suggested'  Approach 

Test  Program  Sets  (TPS)  represent  the  greatest  non-recurring 
Investment  associated  with  automatic  test  equipment.  These  TPSs, 
once  fielded ,  continue  to  perform  tests  in  the  same  sequence  with 
the  same  diagnostic  logic  unless  they  are  modified  through  a 
costly  and  time  consuming  process.  As  a  result  TPSs  are  only 
modified  (1)  to  correct  errors  in  the  program,  (2)  in  response  to 
modifications  in  the  unit-under-test  or  (3)  in  response  to 
modifications  in  the  test  system. 

There  is  another  class  of  information  available  which  could  be 
used  to  improve  the  efficiency  of  the  TPSs.  As  a  result  of  having 
run  TPSs,  the  test  system  develops  out-of-tolerance  data, 
identifies  faulty  subassemblies  and  components,  and  could  develop 
prognostic  trends  from  test  results  on  like  items  tested.  An 
expert  technician  would  use  this  test  history  to  alter  the  test 
sequence  to  look  for  likely  failure  sources  first  or  by-pass 
lengthy  tests  which  have  been  ineffective  in  identifying  faults. 

In  short,  he  would  learn  to  make  the  test  process  more  efficient 
approach  x 

-  Choose  a  sample  of  ten  ATE  test  programs. 

-  Examine  the  go-chain  and  where  in  the  go-chain  the  diagnostic 
logic  is  initiated. 

-  Assume  a  skewed  distribution  of  failures  -  different  from  that 
on  which  the  TPS  development  was  based. 

-  Develop  software  which  recognizes  failure  'epidemics'  and 
modifies  TPS  accordingly. 

-  Examine  test/diagnostic  efficiency  improvement  accomplished  by 

adaptive  program.  j 

Estimated  Time  to  Completion  and  Cost 

One  year  -  two  man-years  engineering 
-  $5K  computer  time 

Impact  on  Operational  Readiness  ! 

(1)  Advantages  1 

i 


Improved  Test/diagnostic  productivity 
Potential  Introduction  of  prognostic  capability 


(2)  Disadvantages 

-  Increases  TPS  software  complexity 
•.  Technological  Risk 

•  Adaptive  software  will  be  inherently  difficult  to  debug. 
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3231  HANOVER  STREET  •  PALO  ALTO.  CALIFORNIA  •  94304 

4  March  1983 


Department  o£  the  Air  Force 
Headquarters  Rome  Air  Development  Center 
Mr.  Anthony  Coppola 
RADC/RBET 

Griff iss  AFB  NY  13441 
Dear  Mr.  Coppola: 

This  is  in  response  to  your  letter  of  7  February  1983,  request¬ 
ing  our  participation  on  a  committee  to  recommend  Artificial 
Intelligence  applications  to  maintenance  and  testing.  We  are 
pleased  to  participate  in  this  exercise. 

Lockheed's  Palo  Alto  Research  Laboratories,  and  other  parts  of 
Lockheed  Missiles  &  Space  Company,  are  studying  AI  applications 
in  the  areas  of  expert  systems  for  field  maintenance  and  test 
of  military  systems  and  tactical  data  fusion ; image  understanding 
for  smart  sensors  and  guidance/navigation;  automation  ■. 
of  manufacturing,  inspection,  and  software  development  and  doc¬ 
umentation;  and  computer  assisted  instruction.  We  feel  therefore 
that  we  are  in  a  reasonably  knowledgeable  position  to  comment  on 
the  utility  of  AI  in  an  application  sense. 

The  remainder  of  this  letter  addresses  the  question  of  AI  applica¬ 
tion  to  maintenance  and  testing  of  military  equipment.  It  is  in  the 
format  suggested  by  the  attachment  to  your  letter: 

0  Recommended  AI  Application  to  Maintenance  &  Test 
0  Approach 

0  Estimated  Time  to  Completion,  and  Cost 
0  Impact  on  Operational  Readiness 
0  Technological  Risk 
0  General  Comments: 

0  Potential  Roadblocks  to  Development  &  Application 
0  Speech  Recognition  &  Synthesis  for  Greater  Friendliness 
0  Use  of  Medium-Expert  Systems 

0  Use  of  the  Maintenance  Expert  System  as  an  Aid  to  O.R. 

0  Other  Lockheed  Investigations  Related  to  Expert  Systems 


PALO  ALTO 
RESEARCH 
LABORATORY 


Briefly,  we  recommend  that  DoD  promote  the  development  and 
application  of  generic  expert  maintenance  systems  to  a  wide 
variety  of  military  hardware.  This  is  because  of  the  very 
nature  of  the  DoD  maintenance  problem,  namely,  many  similar 
items  of  a  particular  equipment  class,  in  geographically  diverse 
sites,  need  to  be  maintained  in  a  high  degree  of  readiness  with 
a  fairly  unskilled  labor  force.  Additionally,  with  the  manpower 
limits  faced  by  the  services  this  labor  force  must  be  as  productive 
as  possible.  The  answer  would  seem  obviously  to  be  the  availability 
of  diagnostic  and  maintenance  experts  to  help  the  field  technician 
over  his  major  hurdles.  Thus,  what  is  called  for  is  a  skilled  set  of 
technicians  who  are  universally  available,  have  the  most  updated 
knowledge,  and  do  not  add  significant  manpower  or  other  elements 
of  cost.  A  computer  based  expert  system  satisfies  these  criteria, 
as  it  is  relatively  cheap  (its  development  costs  are  amortized 
over  the  number  of  similar  types  of  equipment  and  it  could  run  in 
a  machine  like  LISA),  can  be  easily  updated,  can  be  as  good  as 
required,  and  provides  uniform  standards.  Although  the  first  such 
systems,  tailored  to  a  few  specific  items  of  equipment,  will  be 
rather  expensive,  their  generic  nature  will  make  the  subsequent 
tailoring  to  other  specific  items  straightforward  and  cheap. 

It  is  quite  timely  for  such  activity  to  begin  now.  Expert  systems 
have  already  proven  cost  effective  in  medical  (especially  cancer) 
diagnosis  and  treatment  (Stanford  University  and  NIH),  and  in  oil 
exploration  (Exxon),  and  will  be  employed  later  this  year  for 
troubleshooting  malfunctioning  diesel-electric  locomotives  (GE). 

The  impact  of  such  a  maintenance  approach  can  be  measured  in  terms 
of  operational  readiness,  where  it  can  result  in  lower  skill  and 
training  requirements,  lower  manpower  requirements,  higher  morale, 
and  a  general  improvement  in  the  speed  and  accuracy  of  trouble¬ 
shooting  and  fault  finding.  (These  are  items  2,3,  and  6  of  the 
Technology  Selection  Criteria  contained  in  John  Rivoire's  letter 
to  me  of  18  January  83.)  In  addition,  it  is  conceivable  that  a 
variant  of  the  noeizttezsmae  expert  system  can  be  used  as  an  aid  in 
operations  research  to  improve  the  capability  to  project  and  analyze 
the  impact  of  reliability  and  maintenance  (design/cost)  on  operational 
readiness.  (See  under  General  Comments  below  —  this  is  item  11  of 
the  Technology  Selection  Criteria.) 

RECOMMENDED  AI  APPLICATION  TO  MAINTENANCE  AND  TEST: 

We  recommend  that  the  DoD  promote  the  investigation,  development, 
and  application  of  expert  system  methods  to  the  maintenance  and 
test  of  broad  classes  of  military  equipment  and  systems  of  interest, 
and  the  use  of  expert  systems  that  are  tailored  for  specific  equip¬ 
ment  as  a  training  aid  for  maintenance  technicians.  At  LMSC  we  are 
considering  applying  such  methods  to  field  maintenance  of  Navy 
Trident  and  Army  RPV  —  military  systems  we  are  most  familiar  with  — 
and  will  eventually  be  evaluating  the  potential  of  applying  these 
methods  to  military  aircraft  with  our  sister  company,  Lockheed 
Georgia.  Use  of  the  expert  system  as  part  of  a  training  module  is 
being  considered  as  well. 


APPROACH: 


Attachment  1  describes  the  Lockheed  Expert  System  (called  LES) 
in  text  form;  Attachment  2  is  a  viewfoil  distillation.  LES  is, 
roughly  speaking,  divided  into  a  friendly  inference  engine 
and  an  independent  knowledge  base.  Currently,  LES  is  "written" 

(the  inference  engine  is  complete),  but  "empty"  (the  knowledge 
base  is  blank  as  far  as  military  systems  go).  To  give  a  specific 
example  of  its  use,  but  to  avoid  classification  problems,  we  have 
demonstrated  its  use  for  car  maintenance  and  repair  by  filling  the 
knowledge  base  from  a  standard  GM  car  repair  manual  —  this  example 
can  be  seen  in  Attachments  1  and  2.  The  system  is  coded  in  PL/I, 
which  is  a  more  standard  and  efficient  language  than  LISP,  and  has 
better  string  structure  than  PASCAL  or  FORTRAN.  It  is  thus  quite 
portable,  and  can  be  fitted  into  an  8086  microprocessor.  However, 
there  is  no  reason  that  the  system  could  not  readily  be  convert¬ 
ed  to  one  of  the  more  accepted  AI  languages  such  as  LISP  or  a 
LISP  variant,  or  PROLOG. 

ESTIMATED  TIME  TO  COMPLETION,  AND  COST: 

For  a  typical  application  on  a  system  like  Trident,  we  estimate 
that  about  five  man-years  total  effort  and  roughly  $100K  of 
computer  cost  would  be  required  to  fill  the  knowledge  base  with 
Trident  parameters  and  verify  that  the  procedures  meet  specs ; 
this  effort  and  cost  would  probably  be  distributed  over  two 
years.  (Exact  effort,  cost,  and  time  would  depend  on  the  scope  of 
the  application.)  Although  it  is  difficult  to  say,  we  would 
guess  that  roughly  the  same  effort,  cost,  and  time  would  be  re¬ 
quired  for,  say,  C-5  or  Hercules.  Cost  and  effort  for  RPV 
probably  would  be  half  that  for  Trident,  but  would  have  to  be 
distributed  over  a  longer  period  since  the  RPV  is  not  as  mature 
or  field-tested,  and  a  maintenance/ test  methodology  is  not  fully 
developed  yet. 

IMPACT  ON  OPERATIONAL  READINESS: 

Relative  to  maintenance  performed  using  conventional  paper  manuals, 
we  expect  use  of.  the  friendly  expert  system  to  decrease  costs, 
increase  technicians  accuracy  and  speed ,  and  improve  morale .  LES 
draws  conclusions  from  the  answers  that  the  techician  provides 
to  its  questions,  and  if  questioned  itself  can  explain  the 
reasoning  process  used  to  arrive  at  its  conclusion  by  exhibiting 
in  English  text  the  appropriate  sections  of  its  extensive  if-then 
program  if  low.  Thus,  in  addition  to  being  considerably  faster  and 
less  tedious  than  skipping  through  a  hefty  manual  of  procedures 
for  each  of  a  number  of  maintenance  protocols,  it  can  increase 
the  confidence  of  the  technician  in  the  (his)  diagnosis  and 
possible  treatment  of  problems,  and  can  educate  him  to  the  "logic 
and  physics"  of  the  connections  between  certain  symptoms  and 
certain  parts  of  the  equipment  under  test.  LES  relates  faults 
to  test  results  or  to  characteristics  of  the  equipment,  as 
appropriate,  and  can  spell  out  the  implications  and  seriousness  of 
test  results  or  equipment  problems.  When  a  piece  of  analysis  has 


reached  a  dead  end,  LES  can  request  specific  additional  infor¬ 
mation  from  the  user  and  tell  the  user  why  it  needs  it.  LES  keeps 
a  running  status  of  the  diagnosis,  and  can  itself  be  updated  and 
upgraded  readily;  and  if  the  knowledge  base  is  created  and  enriched 
by  several  experts  over  a  period  of  time  (by  the  addition  of  rules), 
the  expert  system  can  become  more  thorough  than  any  single  human 
expert.  The  net  result  of  improved  technician  accuracy,  speed, 
and  morale  (through  real-time  "education"  as  well  as  relief  from 
paper-flipping  tedium)  will  be  increased  operational  readiness  per 
dollar. 


TECHNOLOGICAL  RISK: 


Technological  risk  associated  with  LES  application  to  maintenance 
and  test  appears  to  be  very  small.  Division  of  LES  into  an  in¬ 
ference  engine  and  a  knowledge  base  means  that  the  inference  engine 
portion  is  not  system  peculiar  —  essentially  only  the  knowledge 
base  has  to  be  changed  for  LES  to  be  applicable  to  other  particular 
military  systems,  so  the  maintenance/test  use  capability  appears 
to  be  quite  generic.  Because  of  the  extensive  use  of  "if-then" 
rules,  the  Writing  of  new  knowledge  bases,  starting  from  an  already 
prepared  paper  manual,  is  quite  straightforward.  LES  is  written 
in  PL/I  G  (  General  Purpose)  subset,  proposed  ANSI  Standard  BSR 
X3-74;  thus  it  is  portable,  and  easy  to  interface  to  other  pro¬ 
grams  in  FORTRAN  or  PASCAL.  (It  would  have  been  more  difficult 
to  develop  in  Ada  since  standard  Ada  does  not  have  a  character 
string  variable,  but  it  can  be  machine  translated  to  Ada  if  necessary. ) 


GENERAL  COMMENTS: 


0  Potential  Roadblocks  to  Development  &  Application  —  AI  technology 
and  microprocessor  availability  are  now  at  the  level  where  applica¬ 
tion  of  expert  systems  to  maintenance  of  complicated  military  hard¬ 
ware  is  a  realistic  near-term  goal.  Indeed,  expert  systems  have 
already  proven  cost  effective  in  medical  (especially  cancer) 
diagnosis  and  treatment,  and  in  oil  exploration,  and  are  to  be 
employed  later  this  year  for  diesel  electric  locomotive  repair. 
Currently  a  potential  roadblock  to  aggressive  investigation  of 
this  capability  in  the  military  arena  may  be  the  skepticism  of 
military  managers  about  these  techniques  and  their  value.  Indeed, 
it  is.  very  difficult  in  abstract  discussion  to  explain  why  "some¬ 
thing  that  an  18-year-old  can  do  by  wading  through  a  manual"  can 
perhaps  be  more  effectively  done  using  a  computerized  expert 
system.  We  believe  that  the  live  demonstration  of  a  few  specific 
working  models  —  even  in  trivial  applications  like  car  repair  — 
is  far  superior  to  abstract  discussion  for  conveying  an  under¬ 
standing  and  appreciation  of  what's  new,  different,  and  potentially 
better  about  this  maintenance  approach  than  previous  ones .  And 
there  will  undoubtedly  be  resistance  within  the  defense-related 
industries  to  take  the  initiative  and  commit  to  full  scale  develop¬ 
ment  and  application  of  the  expert  systems  approach,  until  the 
military  managers  are  convinced  of  its  promise  and  simplicity  — 
perhaps  to  the  point  w!  re  they  begin  raising  standards  and  ask 
for  computerized  maintenance  manuals  to  accompany  new  systems 
and  equipment. 
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0  Speech  Recognition  and  Synthesis  for  Greater  Friendliness  —  it  is 
not  premature  to  be  considering  as  well  the  possibility  of  incorp¬ 
orating  into  the  expert  system  such  other  near-term  Al-related 
technologies  as  speech  (spoken  number/yes/no/clear/repeat/next/ 
why,  etc.)  recognition  to  free  the  technician's  hands,  or  speech 
synthesis  to  free  the  technician's  eyes.  There  are  already 
available  from  such  outfits  as  Interstate  Electronics  and 


Telesensory  Speech  Systems  speech  recognition  modules  and  speech 
synthesis  modules  of  highly  impressive  capability  that  can  be 
plugged  right  into  terminals.  However,  there  is  still  a  surprising 
amount  of  skepticism  about  the  maturity  of  this  technology  on  the 
part  of  people  who  have  actual 2v  used  it,  and  view  of  this  we 
would  recommend  that  the  advertisement  of  such  "futuristic  bells 
and  whistles"  be  kept  to  a  minimum  during  the  live  demonstration 
referred  to  in  the  previous  paragraph,  unless  the  modules  are 
present  on  the  expert  system  being  demonstrated.  In  any  case, 
it  should  be  clear  that  the  intelligence  and  expertness  of  the 
system  is  independent  of  this  optional  enhancement  of  user 
friendliness. 

0  Usefulness  of  Medium-Expert  Systems  ~  An  expert  system  need  not 
even  be  terribly  "expert"  before  it  can  be  useful.  For  example, 
use  of  a  terminal  with  a  computerized  procedure  map  can  consider¬ 
ably  lower  the  risk  of  error  due  to  losing  one's  place  on  a  bed- 
sheet-sized  procedural  flow  chart  or  any  of  a  number  of  subordinate 
charts;  use  of  a  system  that  can  record  and  list  the  paths  follow¬ 
ed  and  remedies  performed  can  considerably  reduce  the  time  and 
tedium  associated  with  the  writing  of  maintenance  reports. 

0  Use  of  the  Expert  System  as  an  Aid  to  Operations  Research  —  It 

is  conceivable  that  by  incorporating  figures  relating  to  individual 
component  MTBFs  and  individual  component  reliability /cost  correla¬ 
tions  into  a  subset  of  the  maintenance  expert  system's  knowledge 
base,  an  operations  analyst  would  be  able  to  use  a  variant  of  the 
expert  system  to  rapidly  (automatically)  explore  a  plethora  of 
interrelated  hardware  reliability-vs.-cost  trades,  measuring  the 
impact  of  each  on  operational  readiness  and  net  (hardware  plus 
maintenance  and  repair)  cost,  until  he  arrives  at  an  optimal 
point  of  minimum  net  cost  for  a  given  required  level  of  operation¬ 
al  readiness.  At  Lockheed  we  are  only  beginning  to  investigate 
this  possible  application  of  a  maintenance  expert  system. 
(Incidentally,  some  interaction  with  operations  research  personnel 
is  highly  desirable  during  the  development  of  the  maintenance 
expert  system  anyway,  since  it  is  handy  for  the  expert  system  to 
have  some  idea  of  the  likelihood  of  a  particular''  fault  and  some 
idea  of  what  is  easy  to  test  for  in  order  for  it  to  guide  the 
technician  in  the  most  cost-effective  manner. ) 

0  Other  Lockheed  Investigations  Relating  to  Expert  Systems  —  Lockheed 
approaches  to  the  tactical  data  fusion  area  and  to  the  smart  sensor 
and  guidance/navigation  area  also  make  use  of  the  coupling  of  an 
inference  engine  with  a  knowledge  base.  Of  course,  in  the  smart 
sensor  area,  the  system  will  not  have  interface  —  friendly  or 
otherwise  —  with  a  human;  and  in  both  areas,  the  knowledge  bases 
ere  considerably  more  problematical  that  in  the  maintenance/test 


I  look  forward  to  meeting  with  you  at  the  conference  in  April 


Sincerely, 


R.  P.  Caren 
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ARTIFICIAL  INTELLIGENCE  APPLICATIONS  TO 
EQUIPMENT  MAINTENANCE  AND  TESTING; 

ONR  INPUT 


1.  An  A. I.  Based  Expert  System 

a.  Description:  A  computerized  Expert  System  which  provides  advice  to 
humans  on  the  trouble  shoot,  diagnosis,  and  repair  aspects  of  equipment 
maintenance.  Beyond  containing  the  conventional  "if-then  rules",  this  system 
offers  idiosyncration  interaction  or  consulting  help  to  the  human  in  a 
multiplicity  of  forms:  the  user  can  interact  or  have  advice  rendered  via 
speech,  printed  narrative,  graphics,  cartoons,  motion  pictures,  or  animated 
drawings/blueprints.  To  the  concern  with  the  burden  of  completing 
maintenance  forms  reflective  of  the  repair,  this  futuristic  system  casts  up 
on  a  portable  CRT  facimiles  of  all  maintenance  forms.  The  user  substitutes  a 
touch-screen  input  device  for  pencil  annotations;  for  example,  in  the  case  of 
parts  used  in  the  repair,  the  screen  lists  all  parts  (nomenclatures  and  stock 
numbers)  appropriate  to  the  repair.  The  user  touches  the  parts  he  has 
actually  used  and  indicates,  by  touching  numbers  appearing  on  his  screen, 
whether  he  has  used  one,  two,  three,  etc.  of  each  part.  The  system,  then, 
automatically  fetches  and  prints  in  the  appropriate  block  such  data  as  unit 
of  issue,  unit  price,  total  dollar  value  of  all  parts  used,  manufacturer's 
drawing  number,  etc.  Additional  to  advising  on  standard  repairs,  the  expert 
system  consults  on  jury-rigs,  substitute  parts  and  cannibillzation 
candidates.  It  also  automatically  updates  all  appropriate  records,  e.g., 
stock  record  cards,  order  levels.  The  system  of  the  future  is  highly 
interactive  —  permitting  users,  via  their  portable  terminals,  to  input 
comments  on  standard  procedures  or  to  recommend  new  repair  procedures. 

b.  Approach 

(1)  Select  example  high  value  equipments;  value  judged  by  cost, 
military  essentiality,  and/or  concerns  about  excessive  downtime. 

(2)  Establish,  via  knowledge  engineering  interview  techniques, 
appropriate  production  rules  (if-then  statements)  for  the  candidate 
equipments. 

(3)  Explore  video-disc  technology  for  the  man-machine  interaction. 

(4)  Design  portable  terminals  and  test  equipments  for  maintenance 
personnel. 

(5)  Build  databases. 

(6)  Develop  methodology  for  dynamic  validation  of  rules. 

c.  Estimated  Time  and  Coats 

(1)  Time  and  costs  depend  on  number  of  equipments  for  which  expert 
rules  must  be  developed. 
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(2)  Costs  for  a  required  two  to  three  year  applied  R&D  expert; 
approximately  1M  to  1.5M. 

d.  Impact  on  Readiness 

(1)  Advantages 

(a)  Dramatic  improvement  in  the  demonstrated  capabilities  of 
maintenance  technicians. 

(b)  Improved  quality  of  repair  -  speed  up  in  fault  diagnosis 

and  correction. 

(c)  Reduction  in  man-power  requirements. 

(d)  Reduction  in  human  skill  needs. 

e.  Technical  Risks 


(1)  It  may  be  difficult  to  develop  a  template  of  production  rules 
slots  and  procedures  that  apply  over  a  broad  range  of  equipments. 

(2)  Costs  of  acquiring  rules  for  vast  numbers  of  equipments  may  be 
prohibitive. 

(3)  Costs  of  building  the  man-machine  system  (voice,  graphics,  motion 
pictures,  etc.)  tailored  to  each  equipment  will  be  high,  and  may  vary  as  a 
function  of  scale. 

2.  Automating  the  Creation  and  Presentation  of  Technical  Manuals 

a.  Description:  A  methodology  for  generating  both  color  graphics  and 
narrative  to  the  goal  of  automating  the  production  of  technical  manuals  and 
instruction  books.  This  same  technology,  combining  pictures,  ideographlcs , 
and  text,  permits  the  digitization  and  computer  storage  of  technical 
documents  with  the  capability  for  real  time,  remote  terminal  access  of  all 
stored  data  (text  and  pictures). 

b.  Approach 

(1)  Designate  DoD  R&D  agencies  to  replicate  prototype  capabilities 
now  being  demonstrated  at  6.1  funded  university  labs. 

(2)  Conduct  experiments  to  prove  feaslblity  of  a  full  scale 
operational/production  version  of  the  prototype  system. 

(3)  Create  A. I.  reasoning  capability  for  labeling,  titling,  and 
indexing  the  graphic  images,  and  for  generation  and  matching  text  and 
pictures. 

(4)  Develop  rules  for  generalizing  this  capability  across  designated 
categories  of  equipments. 
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c.  Estimated  Tine  and  Costs 

(1)  Three  years  and  1.5  to  2.0M. 

d.  Impact  on  Readiness 

(1)  Improved  quality  of  technical  manuals. 

(2)  Reduced  costs  of  generating  manuals. 

(3)  Via  computerized  store  of  technical  documents  -  the  system  both 

saves  space  of  storing  voluminous  paper  documents  and  offers  real  time,  remote 
access  to  all  required  maintenance  data. 

e.  Technical  Risks 


(1)  Difficulty  of  generalizing  the  capability  across  equipments. 

(2)  Developing  a  methodology  for  assigning  index  terms  to  pictures. 

(3)  Creating  a  concept  of  a  thesaurles  of  index  terms  to  permit 
retrieval  by  users  who  are  not  familiar  with  the  standard  terminology. 

3.  A  natural  Language  Database  Query  System  in  an  Equipment  Maintenance 
Context. 


a.  Description:  An  English  language  inquery  capability  to  an  equipment 
maintenance  database.  This  A. I.  system  could  permit  on  line  computerized 
responses  to  a  large  number  of  factual  questions.  However,  state-of-the-art 
limitations  impose  various  constraints  on  the  initial  Implementation  of  such 
systems:  (l)  responses  confined  to  fairly  simple  factual  questions;  no 
treatment  of  what-if  questions,  (2)  a  limited  vocabulary  or  semantic  base; 
primarily  a  noun  and  verb  world,  and  (3)  avoidance  of  psychological  nuances 
of  languages.  Examples  of  questions  answerable  by  a  state-of-the-art  natural 
language  system:  (1)  Provide  Federal  Stock  Number  for  a  given  Manufacturer's 
plan  and  piece  number,  (2)  How  many  parts  of  a  designated  stock  number  failed 
on  a  particular  aircraft  type;  in  a  bounded  time  period;  over  all  aircraft; 
etc.,  (3)  Dollar  value  of  failed  parts. 

b.  Approach 

(1)  Designate  DoD  agency  to  evaluate  state-of-art  natural  language 

systems. 


(2)  Choose  a  maintenance  domain  or  applications  area  and  build  a 

database. 


(3)  Establish  an  experimental  environment  utilizing  human  military 
officers  and  civil  service  engineering  personnel  to  test  the  system. 

(4)  Select  appropriate  personal  computers/tsrminals. 
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(5)  Collect  data  on  questions  answered  routinely  by  the  experimental 
system  and  those  that  are  either  totally  rejected,  or  where  the  answers  are 
deemed  Inappropriate. 

(6)  Using  results  of  the  experimental  effort.  Implement  production 
system  with  caveats  on  limitations.  Refer  problems  to  the  natural  language 
research  community. 

c.  Estimated  Time  and  Costs 

(1)  Two  years  and  1M. 

d.  Impact  on  Readiness 

(1)  Permits  on-line  exploration  of  many  more  maintenance  Issues  than 
are  possible  without  a  natural  language  Interface. 

(2)  Encourages  direct  Interaction  with  maintenance  data  by  senior 
managers  as  well  as  staffers  and  computer  professionals. 

(3)  Improves  material  readiness  through  immediate  responsiveness. 

e.  Technical  Risks 


(1)  Difficulty  of  Introducing  a  natural  language  system  to  high  level 

managers. 


(2)  Possibility  that  what  is  feasible,  using  current  state-of-art 
technology,  will  not  be  satisfactory  for  operational  practice. 

4.  Machine  Reasoning  in  an  Equipmsnt  Maintenance  Context 

a.  Description:  A  fully  automated  A. I.  reasoning  system  that  offers 
answers  to  truly  complex  questions,  e.g.,  provide  an  explanation  and 
rationale  for  an  observed  series  of  aircraft  crashes  or  equipment  failures. 
Given  the  development  of  an  appropriate  database  (configuration  files, 
failure  data,  maintenance  histories,  and  personnel  files)  this  advanced 
machine  reasoning  system  invents  its  own  search  strategy  to  probe  for  such 
reasonable  possible  answers  as:  (1)  all  of  the  planes  which  crashed  shared 
In  common:  a  missed  or  extended  overhaul;  a  unique  pattern  of  failures  that 
preceded  and  anticipated  the  crash;  a  unique  equipment  configuration;  all 
were  manufactured  consecutively  at  a  given  plant. 

b.  Approach 

(1)  More  support  for  the  basic  research  A. I.  community. 

c.  Estimated  Tima  and  Costs 
(1)  Five  to  ten  years. 
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d.  Impact  on  Readiness 

(1)  Quickly  and  accurately  arrives  at  explanations  of  underlying 
causes  of  major  maintenance  disasters. 

(2)  In  lieu  of  arriving  at  global  answers  iteratively  and  slowly  by 
building  systems  that  provide  one  by  one  responses  to  thousands  of  simple 

questions  -  this  system  goes  directly  and  swiftly  to  the  heart  of  the 

matter  -  provide  me,  the  interrogator,  an  explanation  of  what  might  have 

caused  the  crash  of  those  planes. 

e.  Technical  Risks 


(1)  Reasoning  is,  perhaps,  the  most  challenging  problem  that 
confronts  Artificial  Intelligence. 

5.  Computer  Driven  Crisis  Alerting  System 

a.  Description:  From  the  high  level  logistics  coanand  perspective, 
there  is  a  need  to  develop  "activist"  computer  systems  with  capabilities  both 
for  recognizing  impending  crisis  situations,  and  for  sounding  the  appropriate 
alert  or  alarm  prior  to  their  occurrence.  To  appreciate  the  utility  of  an 
alerting  or  triggering  mechanism,  one  has  only  to  reflect  on  the  passive 
nature  of  the  best  available  computer  systems  on  the  market  today.  Only  to 
the  degree  that  the  right  questions  are  asked  by  the  right  person  at 
precisely  the  right  point  in  time  does  .this  passive  interactive  capability 
support  the  decision  maker's  requirements.  An  alert  capability,  in  contrast 
to  passive  management  support  systems,  is  a  concept  wherein  threshold  or 
alarm  conditions,  initially  assigned  by  knowledgeable  commanders,  are 
continually  machine  monitored  and  compared  with  the  dynamic.  Incoming  data 
stream.  Alert  or  alarm  thresholds  (representing  a  manager's  anticipation  of 
an  impending  or  potential  crisis  situation)  are  relatable  to  almost  all  of 
the  comeand  and  managerial  responsibilities  of  loglsitlcs  deployment. 
Particular  examples  Include:  computer  monitoring  of  the  behavior  of 
equipments  and  complex  interactions  across  equipments;  machine  anticipation 
end  warning  of  increasing  failure  rates  and  the  possible  shortfall  of  spare 
parts;  alerting  on  unanticipated  increases  in  failure  rates  resulting  from 
the  Implementation  of  scheduled  maintenance  policies;  a  warning  on  the 
impending  unavailability  or  delays  in  delivery  schedules  of  logistics 
resources. 

b.  Approach 

(1)  Create  a  database. 

(2)  Involve  a  user  community  in  establishing  logistics  thresholds 
(alert  conditions). 

(3)  Establish  a  dynamic  database  collection  effort. 

(4)  Design  algorithms  for  machine  matching  of  dynamic  data  stream 
against  thresholds. 
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(5)  Develop  (through  experimentation)  insights  on  optimal 
methodologies  for  calling  "alerts"  to  the  attention  of  logistics  commanders 
e.g.,  narrative  messages  printed  at  terminals,  voice  alerts,  graphics 

messages. 


Nr, 


(6)  Build  A. I.  capability  for  system  self  designation  of  thresholds 


c.  Estimated  Time  and  Costs 


,'C 


(1)  Two  years  and  1M 


d.  Impact  on  Readiness 


(1)  An  alerting  system  would  permit  anticipation  and  possible 
prevention  of  impending  logistics  crises. 


e.  Technical  Risks 


(1)  Capability  of  logistics  commanders  to  determine  and  set 
thresholds. 


(2)  Building  a  computational  engine  that  ensures  real  time  pattern 
matching  between  machine  stored  thresholds  and  incoming  dynamic  data  stream 


IVIN  DEN1C0FF 
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February  15,  1983 


Mr.  Anthony  Coppola 

RBET/A.  Coppola/AV  587-4726 

RADC/RBET 

315-330-4726 

Qrlffiss  AFB,  MY  13441 

Dear  Mr.  Coppola: 

I  cannot  participate  in  the  DoO  study  on  AI  applications  to  testing  due  to 
the  press  of  other  business;  however,  I  can  make  some  brief  recommendations 
which  are  philosophical  in  nature,  rather  than  addressed  to  particulars.  My 
experience  is  in  AI,  and  my  group  has  been  engaged  in  building  two  sorts 
of  expert  systems.  These  are:  a  Planner  (used  for  generating  computer 
code),  and  a  Diagnostician  (used  for  locating  faults  in  electro-mechanical 
systems).  We  are  at  present  attempting  to  integrate  the  two  into  a  larger 
system,  a  feedback  loop,  that  will  provide  automated  command  and  control 
with  error  recovery. 

My  view  is  similar  to  that  of  Randy  Schumacher  of  ONR.  It  is  his  opinion 
that  the  next  generation  of  military  hardware  will  be  so  sophisticated  and 
complex  that  the  present  approach  of  ATE,  requiring  the  test  of  every 
possible  fault  and  the  pre-construction  of  fixed  fault-diagnosis  trees, 
will  Inevitably  fail  in  determining  faults  for  the  next  generation  of  hard¬ 
ware.  I  have  heard  statistics  that  indicate  that  even  present  ATE  equip¬ 
ment  doesn't  work  very  well.  To  approach  this  problem  properly  requires 
a  basic  change  In  direction.  My  recommendation  is  to  develop  Diagnostic 
expert  systems  that  approach  the  problem  as  a  trained  human  trouble-shooter 
would,  using  failure  physics  and  knowledge  of  how  the  apparatus  to  be  diag¬ 
nosed  Is  constructed.  That  Is  how  our  present  system  is  Implemented.  It 
also  has  the  advantage  of  being  broadly  general-purpose,  so  that  the  same 
diagnostician  will  diagnose  failures  in  a  spacecraft,  or  the  ground  system, 
or  robotics  problems.  Only  the  knowledge  base  has  to  be  changed  to  describe 
the  system  under  investigation. 

a.  Recommended  DoD  action:  Fund  a  research  and  development  program  to 
largely  replace  ATE  systems  with  expert  AI  diagnosticians. 

b.  Suggested  approach:  Select  a  competent  group  or  groups  willing  to 
undertake  this  development.  Forget  ADA,  use  the  LISP  language,  and 
LISP  machines  for  hardware. 

c.  Estimated  time  and  cost:  Two  years;  five  to  ten  man-years  to  program 
the  expert  systems.  Preparation  of  multitudinous  data  bases  would  take 
longer,  but  would  use  the  same  expert  systems  over  and  over. 


rwnmim  (213)  354-4321 
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Impact 

Advantages:  Would  diagnose  all  the  faults  quickly  diagnosed  by  a  human  and 
could  also  do  the  most  common  difficult  diagnoses.  Would  attack  the  most 
common  problems  that  arise  rather  than  every  conceivable  problem.  Would 
cost  far  less  to  Implement  than  present  ATE  systems  and  would  work  better. 

Disadvantages:  Would  require  the  junking  of  many  existing  AF  systems  in  which 
there  are  large  investments. 


Sincerely, 


L— 


Leonard  Friedman,  Bldg.  278 
Automated  Problem-Solving 
Group  Supervisor 


LF:mb 


TO:  MAJOR  PHILIP  CORM 

e/o  APSC/DLZ  (DR.  KULP) 

Attached  is  an  edited  copy  of  Russ  Genet's  input  to  Bob  Sasmor  for  the 
Training  and  Simulation  (include  Maintenance  Performance)  Work  Group 
of  the  JDL/TIP  for  AI.  Although  the  APHRL/L2  Division  Chief  and  I 
have  read  and  edited,  this  is  submitted  outside  official  channels  and 
has  not  been  submitted  for  the  usual  staff  reviews  or  approval  by 
AFHRL/CC. 

EARL  A.  ALLUISI 
AFHRL/CCN 


ARTIFICIAL  INTELLIGENCE  IN  MAINTENANCE  AND  DIAGNOSTICS 

Russell  M.  Genet 

Logistics  and  Human  Factors  Division 
Air  Force  Human  Resources  Laboratory 


INTRODUCTION 


In  planning  its  research  and  development  (R&D)  programs,  the  Air 
Force  Human  Resources  Laboratory  (AFHRL)  must  consider  the  direction 
and  impact  of  technology  on  the  future  conduct  of  Air  Force 
operations,  as  many  years  usually  pass  between  laboratory  R&D  efforts 
and  applications  of  the  R&D  findings  in  the  operational  Air  Force. 
Thus,  it  is  imperative  that  the  laboratories  conduct  R&D  in  areas  that 
are  appropriate  to  the  technology  expected  to  be  fielded  a  decade  or 
smre  in  the  future.  In  considering  what  R&D  is  appropriate  for  the 
Logistics  and  Human  Factors  Division  of  AFHRL,  thought  needs  to  be 
given  to  the  nature  of  logistical  support  in  the  Air  Force  of  the 
future.  In  this  brief  paper,  the  area  of  equipment  maintenance  is 
addressed  —  particularly,  the  area  of  diagnostics  of  equipment  faults 
at  the  field  level. 

What  trends  can  be  seen  in  Air  Force  equipment  and  its  diagnosis 
that  would  be  helpful  in  predicting  what  this  area  will  look  like  10 
or  20  years  from  now?  There  is  a  trend  towards  increased  use  of 
electronics  rather  than  mechanical  equipment.  A  good  example  of  this 
is  fuel  control  systems  on  jet  engines,  and  another  is  flight 
controls.  There  is  also  a  trend  away  from  analog  electronics  towards 
all  or  almost  all  digital  systems.  Equipments  are  increasingly  using 
embedded  microcomputers  and  "speaking"  to  other  equipments  over 
standardised  busses.  Equipments  on  aircraft  and  missiles  are 
communicating  more  often  with  ground  equipments  via  digital 
communication  channels.  Increasing  use  is  being  made  of 
built-in-test-equipment  (BITS).  As -the  amount  of  BITE  (including 
in-flight  monitoring)  increases,  it  will  doubtlessly  be  standardised 
and  consolidated  over  entire  aircraft  and  other  weapon  systems.  These 
various  improvements  should  enhance  diagnostic  decisionmaking  and 
henee  substantially  reduce  the  currently  high  rates  of  "retest  OKs" 
and  "cannot  duplicates"  (CNDs). 


Already,  a  similar,  and  perhaps  more  extensive  revolution  is  taking  place 
in  the  ways  data  are  handled,  accessed,  and  decisions  made.  It  is  clear  that 
in  the  future,  large  amounts  of  technical  data  will  be  accessed  by  Air  Force 
technicians  via  interactive  computers.  The  technology  to  store,  retrieve,'  and 
transmit  vast  amounts  of  data  digitally  via  computers  is  here  today.  The 
printed  technical  order  (TO)  as  we  know  it  today  will  become  an  anachronism  in 
the  future.  In  the  future  TOs  will  be  produced  and  accessed  via  computers. 
However,  these  computerized  TOs  of  the  future  will  not  be  TOs  as  we  know  them 
today;  the  Air  Force  maintenance  technician  will  be  able  to  react  and  converse 
with  this  computerized  system.  Also,  the  system  will  be  able  to  learn  and 
adjust  its  behavior. 

In  short,  we  can  expect  that  in  the  Air  Force  of  the  future  we  will  be 
mainly  dealing  with  complex  digital  electronic  systems  in  our  maintenance 
tasks,  and  that  we  will  use  "smart"  digital  computers  to  assist  us  in  our 
maintenance  and  diagnostics  tasks.  This  is,  technically,  an  application  of 
Artificial  Intelligence  (AI) . 

Just  as  we  can  expect  the  BITE  electronics  on  the  various  avionics  and 
propulsion  systems  to  become  an  integrated  whole  (and  it  would  be  silly  for 
future  aircraft  to  have  a  bunch  of  separate  BITE  systems),  similarly  we  can 
expect  the  maintenance  information  systems  to  become  integrated.  It  would  not 
be  practicable  to  have  one  system  to  provide  TO-type  information,  another  to 
give  suggestions  on  what  technicians  might  consider  next  in  their  diagnostics 
(an  AI~based  diagnostics  system),  another  to  record  maintenance  actions,  and 
even  another  to  order  replacement  items  from  supply,  etc.  It  will  be 
imperative  that  there  be  a  single  integrated  maintenance  information  system. 

I  know  a  number  of  local  supermarkets  that  have  been  able  to  integrate  their 
pricing,  inventory,  ordering,  etc.,  into  a  single  computerized  information 
system.  It  seems  likely  that  the  Air  Force  will  see  the  advantage  of  such 
integration  also. 

The  combination  of  (1)  consolidated  and  continuously  monitoring  BITE 
systems,  (2)  computerized  and  interactive  use  of  technical  and  maintenance 
data,  and  (3)  applications  of  these  techniques  along  with  artificial 
intelligence  (AI)  should  substantially  improve  maintenance  in  the  Air  Force  of 
the  future.  It  might  be  well  to  keep  in  mind  that  the  application  of  AI  to 
improve  maintenance  diagnostics,  and  hence  readiness  and  sortie  rates,  is  not 
likely  to  be  overlooked  by  other  Air  Forces.  In  an  editorial  in  the 
January  7,  19«3  issue  of  Science.  Philip  H.  Abelson  points  out  that  "One  area 
in  which  the  Japanese  seem  particularly  eager  to  excel  is  artificial 
intelligence. "  Professor  Edward  Feigenbaum  of  Stanford  believes  that  we  'are 
at  the  beginning  of  a  second  computer  revolution  and  that  ultimately 
applications  of  artificial  intelligence  will  become  more  important  that 
numbercrunching.  Calling  to  mind  Pearl  Harbor  he  said,  "At  dawn  we  slept." 

ASTIPICIAL  INTELLIGENCE  IN  MEDICINE  (AIM) 


Presuming  that  future  Air  Force  maintenance  information  and  diagnostics 
systems  will  be  aided  by  computerized  artificial  intelligence  (AI)  systems, 
are  there  any  such  systesw  currently,  even  in  other  areas,  from  which  Air 
Force  researchers  might  be  able  to  learn7  The  answer  is,  of  course,  yes  that 


AI  has  been  applied  already  to  medical  diagnostics.  In  a  very  small  task  for 
AFHRL,  Or.  Ben  Schwartz  took  an  unbiased  look  at  AI  in  Medicine  and  Public 
Health,  and  its  relation  to  future  Air  Force  maintenance  information  and 
diagnostics  systems. 

Pour  artificial  intelligence  in  medicine  (AIM)  projects  emerged  from  his 
literature  search  and  interviews.  Two  of  them  appear  to  be,  if  not  more 
important,  at  least  more  widely  known  than  the  others.  They  are  MYCIN,  an  AIM 
program  developed  at  Stanford  Medical  Center;  and  INTERNIST,  a  product  of  the 
New  England  General.  Hospital  in  Boston. 

MYCIN  is  a  knowledge-based  (expert)  consulting  system  to  assist  in 
selection  of  medication  (specifically  antibiotics)  for  infections  and 
meningitis.  MYCIN  is  primarily  a  therapeutic  consultant  -  the  diagnosis  is 
assumed  already  accomplished  and  is  not  part  of  the  AI  function.  The  decision 
of  an  anti-bacterial  agent  is  complicated.  It  involves  the  infecting  agent 
and  its  suseeptability  to  specific  medication;  site  of  infection;  route  of 
introduction  of  medication;  patient  weight,  metabolism,  and  condition 
(e.g.  immuno  suppression  medication,  trauma);  contraindications  (allergy,  drug 
interactions);  and  several  other  features.  Bach  factor  is  easy  to  deal  with 
individually,  but  the  combination  of  all  is  often  beyond  the  mental 
capabilities  of  the  human  health-care  provider.  MYCIN  serves  as  a  consultant, 
making  recommendations  or  responding  to  suggestions  of  the  providers.  It  is 
capable  of  interaction  with  the  provider.  For  example,  it  is  able  to  respond 
to  the  query  "WHY  DO  YOU  SUGGEST  THAT?"  (this  feature  was  highly  valued  by 
users  in  several  tests) . 

Both  users  and  outside  evaluators  in  double  blind  tests  have  given  MYCIN 
high  ratings  as  an  adjunct  (never  a  substitute)  for  the  human  health-care 
provider. 

INTERNIST  is  a  diagnostic,  rather  than  therapeutic  AIM  system.  It  also  is 
considered  a  knowledge-based  (expert)  consulting  system. 

INTERNIST  begins  with  a  symptom  and  condition  report  supplied  by  the 
physician  in  basically  unstructured  form.  It  matches  information  items 
against  a  variety  of  disease  conditions  for  strength  of  association.  It  then 
returns  to  the  user  to  request  additional  information  in  a  sequence  calculated 
to  be  s»st  effective  in  reducing  the  possibilities.  Like  MYCIN,  it  will 
function  interactively. 

INTERNIST  was  rated  as  a  "diagnostician"  in  tests.  .  It  scored  better  than 
medical  students,  slightly  worse  than  practicing  nonspecialist  physicians,  and 
substantially  worse  than  specialists.  However,  it  was  not  designed  to  be  a 
stand-alone  system  but  rather  a  member  of  the  man-machine  team.  Users 
consistently  report  that  the  system  enhances  their  diagnostic  talents. 

The  rapidly  growing  R&D  and  applications  gives  credence  to  the  claim  that 
AIM  is  a  topic  of  considerable  current*  interest  and  activity,  particularly 
with  respect  to  diagnosis.  A  provocative  question  to  ask  is:  Can  concepts 


and  techniques  developed  in  the  context  of  AIM  be  transferred  to  the  manmade 
but  inanimate  system  (electronic,  mechanical,  etc.)  maintenance  problem 
environment?  Clear  analogies  can  be  drawn  between  medical  diagnosis  and  fault 
diagnosis  for  inanimate  systems.  Both  human  systems  and  inanimate  systems 
deteriorate  over  time,  are  coherent  (i.e.  comprised  of  interacting  multiple 
components),  and  can  be  thought  of  as  fault  tolerant  in  many  problem 
contexts.  The  analogy,  however,  is  not  perfect.  A  human  can  make  transition 
into  an  improved  state  of  health,  whereas  spontaneous  transition  into  an 
improve  system  state  for  an  inanimate  system  is  highly  improbable,  if  not 
impossible. 

The  AIM  systems  considered  by  Ben  Schwartz  were  all  expert-based  systems. 
As  such,  they  can  capture  the  wisdom  of  the  experienced  "master  sergeants"  and 
make  this  experience  available  to  the  new  recruit.  However,  experts  could 
have  cognitive  and  behavioral  biases  that  may  intrinsically  prohibit  them  from 
determining  optimal  or  good  suboptimal  maintenance  procedures  for  large-scale 
maintenance  problems,  and  an  expert  system  could  propagate  such  biases.  It 
seems  likely,  however,  that  eventually  AI  systems  will  be  able  to  blend  the 
best  of  the  expert  master  sergeant's  wisdom  with  an  accumulating  data  base  of 
actual  maintenance  decisions  and  outcomes  in  a  manner  that  will  allow  the  AI 
smintenance  system  to  get  smarter  over  time.  For  this  to  occur,  it  will  be 
necessary,  of  course,  that  maintenance  data  routinely  be  captured  by  the 
system.  This  suggests,  again,  the  close  ties  between  an  integrated 
maintenance  information  system,  the  consolidated  avionics/propulsion/etc., 

BITE  that  feeds  the  integrated  maintenance  information  system,  and  the  AI 
system  that  serves  as  the  interface  between  the  computerized  systems  and  the 
maintenance  technicians,  and  also  serves  as  the  repository  of  the  accumulated 
expertise  of  the  "super  stripers'*. 


ARTIFICIAL  INTELLIGENCE  IN  EQUIPMENT  MAINTENANCE 


While  the  medical  AI  systems  have  obvious  applicability  to  the  maintenance 
of  Air  Force  equipment,  has  AI  been  applied  directly  to  tbe  problem  of 
equipaMMt  maintenance?  Surprisingly,  the  answer  is  a  definite  "yes".  The  AI 
researchers  at  Stanford  that  developed  MTCIN  realized  that  their  expert-based 
approach  had  many  applications  beyond  the  medical  field.  To  facilitate  the  AI 
expert  systems  approach  in  other  areas,  they  generalized  MTCIN  into  an  AI 
system  known  as  EMYCIN  that  provided  the  computerized  framework  for  tbe 
development  of  any  computerized  AI  expert  system,  and  also  serves  as  a  basis 
and  repository  for  capturing  the  necessary  expert  information. 

In  an  in-house  B&D  effort,  the  Boeing  Company  has  taken  EMTCIN  and  applied 
It  to  an  equipment  maintenance  task.  The  equipment  they  chose  for  this 
pioneering  demonstration  was  a  Hewlett  Packard  printer,  as  it  contained  both 
electronic  and  smiehanical  components,  and  there  were  experts  on  the 
maintenance  of  this  system  available  so  that  the  AI  knowledge  base  could  be 
built.  The  first  phase  of  this  in-house  project  has  been  successfully 
completed,  and  the  capabilities  of  an  AI  expert  system  to  aid  in  equipment 
maintenance  has  been  demonstrated.  The  next  step  is  probably  the 
demonstration  of  the  use  of  an  AI  expert  system  in  the  maintenance  of  Air 
Poree  equipment  under  realistic  military  conditions. 


AIK  FORCE  AI  RAD  DEFINITION  EFFORTS 


There  are  three  closely  related  and  supportive  efforts  underway  in  the  Air 
Force  to  identify  what  AI  R&D  should  be  undertaken  with  respect  to 
maintenance,  testability,  and  training.  The  first  is  a  JlOOK  task  from 
AFHRL/LR  to  Systems  Exploration  on  "Artificial  Intelligence  in  Maintenance 
Diagnostics  and  Information  Systems".  The  objective  of  this  task,  which  has 
just  started,  is  to  identify  potential  AI  research  directions  and  projects 
with  good  payoff  prospects.  Such  projects  would  be  expected  to  result  in 
improved  hnman  performance  of  avionic  maintenance  tasks.  The  second,  and 
closely  related  and  coordinated  task  is  from  AFHRL/TU,  and  is  being 
accomplished  by  the  University  of  Denver  Research  Institute  on  trainihg 
applications  of  AI  in  the  Air  Force.  Particular  attention  is  being  paid  to 
technical  training  as  it  relates  to  the  maintenance  of  electronic  equipment. 

A  one-year,  $1S0K  study  is  about  to  be  awarded  from  RADC.  This  study  will 
survey  existing  and  developing  AI  techniques  for  applications  to  testability 
—  particularly  built-in-test-equipment  (BITE).  Emphasis  is  being  placed  on 
those  AX  techniques  capable  of  practical  application  to  testability  programs 
in  the  very  near  future. 


A  COORDINATED  TECHNOLOGY-BASE  DEMONSTRATION  PROGRAM 


As  suggested  earlier,  for  an  Al-based  maintenance/diagnostics  system  to 
work  at  all,  it  needs  to  be  provided  with  information  from  a  consolidated 
aircraft/system-wide  BITE  system,  and  an  integrated  maintenance  information 
system.  The  consolidated  BITE,  integrated  maintenance  information  system,  and 
AI  expert  system  form  an  interacting  triad  that  supports  the  maintenance 
technician  in  efficiently  and  quickly  accomplishing  his  duties.  All  three  of 
these  eleawnts  require  development  in  a  6.2  program  with  initial  demonstration 
in  a  6.3  program.  While  these  three  elements  of  the  triad  must  be  developed 
together  in  closely  coordinated  programs,  it  would  be  useful  to  consider  them 
individually  just  for  the  purpose  of  clarifying  the  individual  pieces. 


A  6.2  AI  DEMONSTRATION  PROGRAM 


A  demonstration  of  AI  on  the  maintenance  of  a  selected  Air  Force 
electronic  digital  system  will  be  the  objective  of  this  program.  The  approach 
taken  will  be  a  combined  rule-based  (expert)  and  causal  technology.  The 
latter  portion  of  the  technology  takes  advantage  of  the  fact  that  there  are 
many  built-in  logical  relationships  in  equipments.  The  first  18  months  and 
$600K  of  the  two  part  program  will  be  to  verify  that  this  technology  will 
work.  The  second  part  of  the  program  would  demonstrate  this  capability  on  a 
specific  piece  of  Air  Force  equipment.  The  second  part  would  take  an 
additional  12  months  and  $900K. 


A  6.2  INTEGRATED  MAINTENANCE  INFORMATION  SYSTEM  DEMONSTRATION 


The  Integrated  Maintenance  Information  System  (IMIS),  is  a  proposal  for 
providing  all  necessary  automation  to  flightline  personnel  in  one  compact, 
user  friendly  device.  IMIS  would  provide  the  interface  between  the 
maintenance  technician  and  the  AI  expert  diagnostic  system.  However,  IMIS 
would  go  beyond  diagnostics  and  also  provide  the  technician  with  technical 
order  information,  training,  supply  information,  aircraft  schedules,  aircraft 
history,  and  maintenance  record  keeping.  The  first  12  month  part  of  this 
exploratory  development  program  would  define  in  considerable  detail  the 
SMintenance  information  that  needed  to  be  integrated  and  define  the  data  base 
requirements  for  the  integrated  system.  This  phase  would  cost  81. 9M.  The 
second  phase  would  involve  developing  a  prototype  IMIS  system.  This  phase 
would  taka  18  months  and  cost  83. 6M.  In  the  third  phase  the  prototype  would 
be  extensively  evaluated  in  a  field  environment  over  a  period  of  10  months 
<81. 6M). 


A  6.2  INTEGRATED  DIAGNOSTICS  PROGRAM  - 


The  Air  Force  philosophy  for  future  maintenance  is  that  every  fielded 
weapon  system  will  have  a  1001  diagnostic  capability.  This  capability  should 
be  provided  by  trained  people  and  automated  systems  in  the  the  most  effective 
combination.  In  addition,  future  fighter  aircraft  will  need  to  be  maintained 
and  turned  for  a  combat  sortie  in  a  minimum  of  fifteen  minutes.  This  will 
have  to  be  accomplished  with  a  minimal  number  of  people  who  are  not 
extensively  trained  nor  experienced  over  the  entire  range  of  possible 
malfunctions . 

To  achieve  these  future  maintenance  goals,  this  exploratory  development 
effort  will  partition  which  tasks  will  be  automated  and  which  tasks  are  best 
suited  to  the  SMintenance  technician  for  diagnostic  maintenance  purposes.  In 
addition,  it  will  determine  what  information  is  required  and  bow  it  can  most 
effectively  be  presented.  It  will  also  insure  that  redundancy  exists  to 
compensate  for  deficiencies  in  the  equipment  of  the  maintenance  technician 
(1001  diagnostics).  This  research  effort  will  consist  of  two  major  phases. 

The  first  phase  will  provide  a  method  to  quantify  what  information  can  be 
automated  and  what  should  be  taught.  This  will  be  weighed  against  operational 
requirements  to  Insure  a  proper  mix  of  combat  maintenance  capability.  The 
estisMted  cost  is  three  years  and  82M.  The  second  phase  will  be  to  design  and 
field  test  a  system  with  technician  support  to  prove  the  100%  diagnostic 
maintenance  philosophy.  This  will  take  two  years  and  81. 2M. 


A  COMPREHENSIVE  6.3  DEMOSTRATION  PROGRAM 


Once  the  basic  technical  feasibility  had  been  established  in  the 
exploratory  development  program,  it  would  be  necessary  to  demonstrate  in  a 
realistic  Air  Force  environment  that  the  concepts  would  work  out  in  practice. 
Ideally  this  should  be  done  on  some  newly  developed  aircraft  system  as  a  test 
bed.  The  development  of  the  consolidated  BITE,  integrated  maintenance 
information  system,  and  AI  expert  system  for  such  a  demonstration  is  estimated 
to  take  three  years. 
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SUMMARY 


The  us*  of  AI  in  *  consolidated  maintenance  information  and  diagnostics 
system  for  the  support  of  Air  Force  equipment  is  not  a  "far  out"  project,  but 
it  is  alswst  certain  to  be  the  basis  of  maintenance  in  one  or  two  decades. 
Thus,  it  behooves  the  Air  Fore*  to  start  research  in  this  area  as  quickly  as 
possible.  That  such  an  application  of  AI  is  practicable  has  been  suggested  by 
its  application  in  medicine  and  by  the  pioneering  research  at  Boeing.  It  is 
particularly  pertinent  that  AFHRL  be  involved  in  this  research  effort,  as  it 
is  at  the  forefront  of  man/machin*  interfacing  and  interaction,  and  it  is  in 
the  area  of  logistics  resources.  AI  expert  systems  take  their  knowledge  from 
humans  and  interact  with  humans.  It  seems  likely  that  this  blending  of  human 
and  computer  resources  will  be  an  increasing  topic  for  research  at  AFHRL. 


ARTIFICIAL  INTELLIGENCE  APPLICATIONS  TO  TESTING 


John  H.  Hinchman 

General  Dynamics  Electronics  Division 
San  Diego,  California 


POSITION  STATEMENT 

All  military  services  are  confronted  with  increased 
training  costs,  reduced  training  budgets  and  a  widening  gap 
between  the  skills  of  entry-level  personnel  and  the 
abilities  required  to  maintain  increasingly  sophisticated 
systems.  Institutional  training  is  minimized,  and  more 
emphasis  is  placed  on  on-the-job  training.  This  requires  a 
greater  reliance  on  built-in  test  equipment  and  organic 
automatic  test  equipment  support.  Unfortunately,  automated 
testing  (built-in  or  off-line)  does  not  unambiguously  fault 
isolate  all  of  the  time.  The  result  is  a  high  rate  (up  to 
30 t)  of  removal  and  replacement  of  non-faulty  assemblies. 
Reduction  of  the  number  of  suspected  faulty  assemblies 
within  an  ambiquity  group  requires  manual  troubleshooting. 
The  manual  troubleshooting  procedure  used  to  fault  isolate 
to  a  single  assembly  typically  involves  a  brute-force  and 
exhaustive  method  of  remove-replace-retest .  This  manual 
troubleshooting  method  is  expensive  in  terms  of  both  test 
time  and  logistical  support.  Our  premise  is  that  a  solution 
lies  in  improving  on-the-job  training  and  job  performance 
aids.  Me  need  to  provide  tools  at  the  job-site  that  will 
enable  maintenance  personnel  to  pick  up  where  automated 
diagnostics  leave  off;  thus  fault  isolation  and  repair  can 
be  completed  in  a  direct  and  efficient  manner.  Our  approach 
combines  enhanced  job  performance  aiding  and  on-the-job 
training  by  means  of  a  computer-based  intelligent  advisor, 
or  troubleshooting  coach.  The  advisor  embodies  the 
knowledge  of  an  expert  maintenance  technician  and  aids  in 
fault  diagnosis  and  fault  isolation.  The  advisor  also  acts 
as  teacher,  having  the  capability  to  explain  recommended 
actions  and  lines  of  reasoning. 

In  recent  years,  knowledge-based  systems  and  system- 
building  tools  developed  by  researchers  in  artificial 
intelligence  have  been  exposed  to  the  general  public.  The 
success  of  programs  that  perform  medical  diagnosis  (MYCIN, 
INTERNIST) ,  develop  a  VAX  computer  configuration  based  on 
customer  requirements  (R1  or  XCON),  or  locate  mineral 
deposits  (PROSPECTOR ) ,  offers  promise  for  other 
applications  where  expert  practitioners  are  in  short 
ropply.  One  of  those  applications  is  the  diagnosis  and 
^  isolation  of  faults  in  electronic  equipment. 
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RECOMMENDED  DOD  ACTION 


A  focused  effort  is  required  to  develop  system-building 
tools  for  knowledge-based  (expert)  maintenance  aids.  The 
software  tools  would  be  used  to  develop  application 
packages  for  training  and  aiding  maintenance  technicians  at 
the  job  site. 

SUGGESTED  APPROACH 

The  unique  architecture  of  most  knowledge-based  systems , 
i.e.,  the  distinct  separation  of  the  knowledge  base  (a  set 
of  declarative  and  procedural  rules)  from  the  inference 
procedure#  makes  them  extensible,  capable  of  handling 
inexact  inference  and  gives  them  the  ability  to  explain 
lines  of  reasoning.  Of  the  current  system-building  tools 
that  may  be  applicable  to  equipment  maintenance,  all 
require  the  services  of  a  knowledge  engineer  to  act  as 
go-between  to  formulate  the  knowledge  of  a  subject  matter 
expert  into  expressions  that  can  be  used  by  the  inference 
mechanism  of  the  system.  This  so-called  knowledge 
engineering  bottleneck  is  costly,  time  consuming  and 
severely  limits  the  number  of  potential  applications.  We 
need  a  means  by  which  the  expert  maintainer  (the  domain 
expert)  can  directly  create  and  modify  an  application 
knowledge  base.  But  we  must  do  more  than  that.  The 
knowledge  of  a  domain  is  based  on  structure  (in  our  case, 
how  equipment  is  put  togther)  and  function  (how  the 
equipment  works ) .  We  should  not  have  to  burden  the  expert 
maintainer  with  the  task  of  defining  the  declarative  and 
procedural  rules  related  to  structure.  The  software  tools 
that  help  to  automate  the  knowledge  acquisition  process 
must  include  friendly  interfaces,  both  for  the  expert 
maintainer  who  participates  in  development  of  an 
application  package,  as  well  as  for  the  maintenance 
technician.  The  software  must  be  self -tutoring  to  obviate 
specialized  training  and  must  possess  a  supportive  query 
mechanism. 

For  widest  use,  the  system-building  tools  should  be 
designed  to  support  training  and  aiding  at  all  levels  of 
repair.  The  final  product  should  be  available  in  Ada  for 
portability.  Prime  item  specifications  should  require  the 
documentation  necessary  to  support  application  package 
development. 

ESTIMATED  TIME  TO  COMPLETION  AND  COST 

The  time  required  to  develop  a  complete  and  supportable 
product  is  estimated  to  be  five  years.  We  expect  that  the 
development  process  would  be  iterative,  with  a  prototype 
available  for  evaluation  after  the  first  eighteen  months  of 
the  project.  The  staffing  is  estimated  at  an  average  of  ten 
profeaaionals  each  year.  The  computing  resources 
(workstations,  other  computer  time)  required  to  support 


this  effort  jure  estimated  at  $1M  over  the  five  year  period. 
Addin?  labor  costs  (e.g.,  $0.8M  each  year),  the  five  year 
project  cost  would  be  approximately  $5M. 

IMPACT  ON  OPERATIONAL  READINESS 

If  aircraft  subsystems  are  down,  the  aircraft  are  down. 
Under  mobilization,  brute-force  maintenance  procedures 
would  be  neither  supportable  nor  acceptable.  Improved 
training  jttd  aiding  at  the  job  site  will  result  in  more 
efficient  maintenance  actions,  with  less  reliance  on 
logistics  support. 

TECHNOLOGICAL  RISK 

Currently,  there  are  several  knowledge-based  systems 
performing  useful  tasks  on  a  regular  basis.  We  know  that 
the  techniques  developed  so  far  work  well  for  specific 
narrow  domains  in  which  the  expert  knowledge  is  neatly 
bounded.  The  depth  and  breadth  of  success  of  these 
techniques  when  applied  to  electronics  maintenance  will  not 
be  known  until  we  try  them  on  a  representative  set  of 
real-world  problems. 

GENERAL  COMMENTS 

University  research  centers  are  not  recommended  as  lead 
organizations  for  this  work.  The  development  recommended 
here  should  be  directed  and  performed  by  maintenance 
practitioners  having  first-hand  knowledge  of  DoD  readiness 
problems.  Support  will,  of  course,  be  required  from 
artificial  intelligence  specialists;  however,  the  emphasis 
must  be  on  application  rather  than  research. 


POSITION  PAPER 


ARTIFICIAL  INTELLIGENCE  APPLICATIONS 


TO  MAINTENANCE  AND  TESTING 


By  Robert  Hong 


-66- 


The  DoO  has  made  a  long-term  commitment  to  enhance  the  avail¬ 
ability  of  defense  weapon  systems  in  both  peace  and  wartime  environ¬ 
ments  and  to  do  so  as  economically  as  possible.  The  objective  of 
this  study  is  to  identify  opportunities  for  improving  readiness  of 
military  systems  by  means  of  artificial  intelligence  (AI)  techniques. 
The  emphasis  is  to  be  on  its  applications  to  maintenance  and  testing. 
The  results  are  intended  for  possible  application  as  appropriate, 
to  new  weapons  systems  and  subsystems  now  in  the  planning  or  develop¬ 
ment  stage. 

Most  of  the  recent  successes  in  AI  are  in  knowledge  based 
systems  (KBS)  or  expert  systems.  Some  theorem  proving  approaches 
such  as  automated  reasoning  have  also  performed  as  well  or  better 
than  human  experts.  In  the  past  few  years,  a  number  of  papers  have 
been  published  on  the  subject  of  AI  applications  to  maintenance  and 
testing.  In  general,  they  are  on  the  right  track  and  I  will  not 
repeat  them  here. 

The  time  frame  for  developing  the  AI  systems  DoD  might  place 
into  operation  is  Important.  Probably  the  earliest  these  new 
systems  will  be  operational  will  be  3-5  years  from  go-ahead.  By 
that  time,  for  various  reasons  such  as  the  Fifth  Generation  Computer 
Project  and  the  VHSIC  Program,  AI  technology  advancement  will  have 
accelerated  significantly  and  available  computational  power  will 
be  several  orders  of  magnitude  greater  than  now.  In  addition, 
several  NON-VON  type  AI  processors  might  be  available  that  will 
overcome  some  of  limitation  in  applying  AI  to  fr»ear)  real -time  problems. 
Many  LISP  type  machines  are  available  now. 

AI,  especially  KBS,  can  be  applied  to  a  wide  range  of  areas  to 
Improve  readiness  and  availability  of  weapon  systems.  In  addition 
to  maintenance  and  testing,  AI  can  be  used  for  CAI  and  optimizing 
logistics.  Before  we  get  specific  regarding  particular  applications, 
several  areas  needs  to  be  addressed.  Some  of  these  are  as  follows: 

o  The  standard  programming  language  to  be  adopted. 

o  The  computer/processor  systems  to  standardized  on.  (Will 
there  be  a  standard?  ) 

o  The  KBS  now  In  development  belong  to  the  second  generation. 
The  relatively  simple  organization  of  such  first  generation 
KBS  as  MYCIN  is  probably  not  efficient  for  many  of  the 
applications  of  concern  here. 

o  The  number  of  man  hours  required  to  develop  KBS  till  now 
are  quite  large.  Various  tools  are  being  developed  to 
minimize  the  cost  of  KBS.  More  are  needed. 

o  Techniques  are  needed  to  make  KBS  more  user-friendly. 


o  There  is  some  concern  regarding  the  adequacy  of  shallow 
reasoning  to  handle  more  complex  problems. 


The  language  to  be  adopted  may  be  one  of  the  AI  languages 
such  as  LISP  or  PROLOG.  Experience  has  shown  that  PROLOG  Is 
very  much  easier  to  learn  than  LISP.  However,  there  are  other 
tradeoffs.  The  use  of  such  standard  language  as  ADA  would  require 
additional  Instructions  In  Its  repertoire. 

What  computers  are  to  be  considered?  Several  parallel  proces¬ 
sors  for  AI  are  being  developed.  The  use  of  associative  memory 
and  processors  have  been  evaluated  for  this  purpose  by  Grumman 
and  others. 

An  examination  of  the  internal  organization  of  the  first 
generation  KBS  show  that  the  sophistication  of  their  capacity 
to  represent  most  of  what  they  deal  with  is  extremely  limited. 
Almost  universally,  important  notions  such  as  the  relations 
between  hypotheses,  degree  of  beliefs,  models,  temporal  relation¬ 
ships,  diagnostics,  plans,  etc.,  are  either  missing  altogether 
or  else  represented  by  very  simple  medianisms  that  miss  much 
of  the  subtlety  of  human  reasoning.  The  second  generation  KBS 
as  a  minimum  are  developing  new  representational  techniques 
to  overcome  these  limitations. 

Various  tools  such  as  that  for  the  difficult  task  of  knowledge 
acquisition  have  been  developed.  Several  automated  programming 
systems  are  being  developed.  These  range  from  programming  assis¬ 
tants  (apprentice),  to  knowledge  programmers,  and  finally  to 
fully  automated  systems.  These  should  allow  the  domain  expert 
to  provide  knowledge  directly  to  the  expert  system  without  the 
need  of  an  AI  expert.  The  result  should  be  a  much  lower  develop¬ 
ment  cost.. 

As  KBS  are  being  applied  to  more  sophisticated  problems, 
there  will  most  probably  be  a  need  to  Incorporate  deep  reasoning 
capability  to  complement  shallow  reasoning.  This  would  include 
aspects  such  as  causality,  physical  (first)  principles,  theorem 
proving,  and  non-monotonlc  reasoning. 

In  addition  to  automated  knowledge  acquisition  and  automated 
programming.  Improved  man-machine  Interface  with  KBS  should 
be  steps  to  making  the  kBS  more  user-friendly. 

With  the  above  as  a  basis,  the  following  are  the  specific 
recommendations: 

1.  a)  Recommended  DoD  action 

Design  ATG  for  at  least  two  level  of  complexity 
In  fault  diagnosis.  One  for  routine  cases  and 
the  other  for  more  challenging  failures.  It  will 
also  have  self  Improvement  and  possibly  some  learning 
capabilities.  This  will  be  for  digital  systems. 


b)  Suggested  approach 


Use  shallow  reasoning  for  routine  cases .  For  the 
more  challenging  failures,  use  shallow  and  deep 
reasoning.  Grumman  LOGOS  /  DATPG  to  form  basis 

for  this  work. 

c)  Estimated  time  to  completion  and  cost  (dollars 
or  man-years) 

For  a  demonstration  system,  and  assuming  availability 
of  tools.  2  years,  $200,000  computer  costs  and 
6  man  years  of  effort. 

d)  Impact  on  operational  readiness 

(1)  Advantages  -  significant  reduction  In  number 

and  skill  level  of  personnel.  Improve  testability. 

(2)  Disadvantage  -  requires  more  computer  through 
put  and  memory. 

a)  Apply  KBS  to  design  of  VLSI  with  BIT  and  testability 
directly  on  each  chip. 

b)  Use  recently  developed  expert  assistant  for  VLSI 
design  by  Mark  Stefik  and  Lynn  Conway  of  Xerox  PARC. 

Then  develop  equivalent  (hierarchical)  KBS  BIT 
with  capability  for  testability  on  each  chip. 

c)  For  one  type  (microprocessor)  demonstration  VLSI, 

1  yr.:  $100,000  computer  costs,  and  3  man  years 
of  effort. 

d)  (1)  Advantage  -  significant  reduction  in  development 

man-hours.  Provide  building  blocks  of  fully  testable 
chips  for  airborne  and  ATE  equip,  thus  improve 
readiness. 

(2)  Disadvantage  -  inslgnlfcant. 

a)  Develop  KBS  to  diagnose  jet  engine  failures. 

b)  Apply  both  shallow  and  deep  reasoning  approaches. 

Use  applicable  techniques  developed  for  nuclear 
plant  fault  prediction  and  prevention. 

c)  2  yrs.:  $200,000  computer  costs  and  12  man  years 
of  effort. 

d)  (1)  Advantages  •  Capability  to  predicting  and  prevent 

failures.  Ability  to  structure  maintenance  to  the 
actual  needs  of  the  operating  equipment  thus 
reducing  maintenance  manhours. 

(2)  Disadvantages  -  Requires  more  processing 
capability  and  memory. 


a)  Develop  KBS  for  situation  assessment/resource 
allocation  for  spares. 

b)  Apply  mul ti -membership  hierarchical  type  KBS  empha¬ 
sizing  knowledge  representation  and  elicitation. 

c)  2  yrs.:  $200,000  computer  costs  and  6  man  years 
of  effort. 

d)  (1)  Advantage  -  Reduces  support  tail  (reduce 

parts  count,  improve  efficiency.) 

(2)  Disadvantage  -  Requires  use  of  powerful  computer. 

a)  Develop  KBS  CAI  to  provide  training  for  ATE  users. 

b)  Use  video  game  approach  and  teach  personnel  the 
latest  learning  techniques  suggested  by  cognitive 
psychologists  (e.g.  from  CMU). 

c)  2  yrs.:  $200,000  computer  costs  and  6  man  years  of 
effort. 

d)  (1)  Advantages  -  Significant  improvement  in  skill 

level  and  motivation  to  support  personnel. 

Reduce  requirement  for  instructors. 

(2)  Disadvantage  -  Additional  CAI  equipment. 

a)  Develop  parallel  computer/processor  for  KBS  in  military 
appl ications. 

b)  Take  advantage  of  associative  memory/processing.  Use 
CAD  (e.g.  SCALD). 

c)  2  yrs.:  $200,000  computer  costs  and  6  man  years 
of  effort.  $3M  for  hardware. 

d)  (1)  Advantages  -  Significantly  greater  processing 

capability  for  KBS. 

(2)  Disadvantage  •  Additional  processor. 

a)  Develop  ATG  KBS  for  fault  diagnosis  of  analog  circuits. 

b)  Evaluate  the  applicability  of  signal -to-symbol 
transformation,  especially  for  RF.  See  Footnote  Page  5. 

c)  2  yrs.:  $200,000  computer  costs  and  6  man  years 
of  effort. 

d)  (1)  Advantages  -  Significant  reduction  In  processing 

requirement. 

(2)  Disadvantage  -  Insignificant. 


8. 


m 


a)  Automate  the  total  life  cycle  KBS  software  effort. 

Develop  KBS  workstation. 

b)  Take  advantage  of  work  in  progress  at  Kestral  Institute, 
NIT,  Rand  Corp.,  Schl uraberger ,  ISI,  etc. 

c)  2  yrs.:  $400,000  computer  costs  and  24  man  years  of 
effort. 

d)  (1)  Advantages  -  Minimize  one  of  the  major  cost  elements 

in  KBS. 


(2)  Disadvantage  -  Major  undertaking. 

a)  Develop  KBS  techniques  for  in-flight  perform  monitor  / 
on-board  checkout  system  for  both  avionics  and  non¬ 
avionics  which  could  also  be  part  of  graceful  degradation 
and  self-organizing  system.  Reduce  false  alarm  rate. 

b)  Use  both  shallow  and  deep  reasoning  techniques. 

c)  2  yrs.:  $200,000  computer  cost  and  6  man  years  of 
effort. 

d)  (1)  Advantages  -  Improve  mission  success  and  reduce 

false  alarm  rate.  Improve  flight  safety  as  a 

result  of  monitoring  overstress  conditions,  etc.  . 
(2)  Disadvantage  -  More  processing  and  memory  required. 


Footnote 

AI  search  techniques  will  be  employed  on  graph 
theoretic  circuit  representations  In  order  to 
optimize  test  point  selection  and  enhance  testability 
(adaptively,  if  necessary). 

Thereafter,  data  driven  algorithmic  formulations  based 
on  modern  circuit  theory  will  actually  locate  failure 
sources. 
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